NVIDIA, DeepSeek and the Evolution of AI

NVIDIA, DeepSeek and the Evolution of AI

Anu Rajakumar: Since the launch of ChatGPT, artificial intelligence has moved from being a niche technology to becoming the epicenter of innovation, driving growth in semiconductors, reshaping global industries, and even redefining economic productivity. As a critical driver for companies and economies alike, how should investors navigate the opportunities and challenges that AI presents? Is the AI-related volatility seen after DeepSeek's disruptive developments, a short-term phenomenon, or a long-term structural feature of the markets?

Finally, how might geopolitics influence AI and the semiconductor industry? My name is Anu Rajakumar, and joining me today is Jamie Zakalik, Senior Research Analyst and one of Neuberger Berman's leading semiconductor researchers who's here to help us unpack the complexities of AI innovation, semiconductor market trends, and the exciting opportunities that lie ahead. Jamie, welcome back to the show.

Jamie Zakalik: Thanks so much for having me.

Anu Rajakumar: Jamie, let's start by talking about how markets are currently thinking about AI. What's driving investor optimism or skepticism around the space at the moment?

Jamie Zakalik: Taking a step back, I think it's helpful to think about where we've come over the last two years. AI feels like it's one of these super important topics in the world today, but before ChatGPT came out in November of 2022, real-world AI use was much more behind the scenes, not really a household discussion topic. Tangible use cases were in applications like recommender systems or what people on social media might refer to as their algorithm.

This human-like intelligence, these AI’s that generate content, this was more niche, it was more futuristic. Since ChatGPT, that's obviously changed because generative AI is not this far-off futuristic idea. It's happening now. People had the opportunity to interact with it. The response from the public was clear. This is cool. This is useful. This is relevant. It has vast implications for industry and society.

Every large company and a lot of small companies too, they've recognized that being a leader in this next wave of technology, this generative AI era, it's of existential importance, and so as a result, investments have been swift. They've been massive. If you look at the CapEx investments of just the four top US-based cloud and AI companies, CapEx has grown from a cumulative $150 billion in 2023. This year, expectations are north of $300 billion, so doubling in two years.

From a semiconductor perspective, these companies are the picks and shovels. They're the core building blocks of this AI infrastructure. If you look at data center related revenues for a company like NVIDIA, it's grown from $50 billion in 2023 to expectations now are somewhere in the $180 billion this year, so more than tripling. Just as a precursor to the rest of the conversation, one thing the semiconductor industry is known for is its cycles. You've seen time and time again in various semiconductor markets, periods of strong demand where supply gets tight and lead times get extended and customers build inventory. That's the upcycle.

Then it leads to a drawdown where semiconductor companies see outsized weakness as inventories are worked down. One of the key debates for AI semis and the AI investment cycle today is, are these secular growth drivers around AI enough to support this level of AI investment spending and continued growth from here, or is this the usual semiconductor playbook where companies are overbuilding because they'd rather have too much than too little? At some point, we hit the drawdown phase.

Anu Rajakumar: I think that's a great transition to the next question which has really been on folks' mind for the last few weeks. When DeepSeek's R1 model announcement came out recently, that really shook things up in the AI world. You spoke about those cycles, kinds of ups and downs. Can you walk us through the market's reaction to that announcement from DeepSeek and explain what that tells us about the volatility and the risks associated with these types of investments?

Jamie Zakalik: Yes. The DeepSeek announcement sent shockwaves through the market, not just NVIDIA, but a lot of different companies. Maybe just starting, what is DeepSeek? What did they announce? It was pretty much this unknown Chinese AI lab. What they did is they released these two models, V3 and R1. R1 was their reasoning model, which was very competitive with top US models like OpenAI's o1, which at the time of R1's release was one of the best AI models out there. It still is. A few days after DeepSeek came out, OpenAI actually released O3, which is now even more state-of-the-art, but the model was very impressive versus what people considered to be the best of the best.

A Chinese startup putting out a model that is head-to-head with leading US AI companies, this was shocking. The most inflammatory claim was that the cost to build this model was something in the $5 million to $6 million range, not several hundred millions or even billions that US companies have been spending on training models, and they were able to do this using NVIDIA's toned-down H800 chip as opposed to the flagship H100s which have better performance but aren't allowed to be shipped into China.

How were they able to do this? They were constrained by regulation, so they couldn't get the best chips. They had to figure out ways to do more with less. They worked into their training process several algorithmic innovations that weren't necessarily unknown, but they have risks of potentially impairing performance. DeepSeek was able to do it, and keep performance, and keep precision.

I don't want to bore you or any of our listeners, but at a high level, they just used several changes that allowed them to use less memory, less compute, and improve efficiency of model training. Then on top of all of that, they put out this model for the world to use and they charged something around 3% of what OpenAI was charging for a similar model.

Anu Rajakumar: Now, Jamie, why don't you tell us about, how did the market respond to all of the above?

Jamie Zakalik: If you remember, at the start of this discussion I just went through how these large US hyperscalers are spending 300 billion with a B, billion, on CapEx. Obviously, not all will be AI infrastructure, but it's a large investment. This unknown Chinese startup more or less caught up with $5 million. You don't need to be a rocket scientist or maybe an AI engineer, rather, to figure out that there were going to be some questions about why in the world these large companies are spending so much money on NVIDIA hardware, and on power, on networking, on all these different parts of the AI data center landscape if they could do it with a lot less.

While investors were figuring out the answer to that question, there was this sell first figure things out later response. NVIDIA, one of the world's largest companies, fell by 17% in one day. That wiped out roughly $600 billion of market cap. Just to put that in perspective, that's three times the total market cap of McDonald's. Huge loss of market cap in one day, and a lot of adjacent companies that supply power or networking, they were down 20% or 30% in one day.

It was quite shocking, but it is a clear representation that there's a lot of skepticism about AI, about this investment cycle, about whether or not we're in a bubble-like environment, from the 2000, 2001 timeframe. That is not my view, but it's not uncommon for people to call me up and ask me how NVIDIA is different than CISCO from 2000. People are scared, they're skeptical, and the market is quite narrow in outperformance, especially in the semiconductor space. The moves have been quite extreme.

Anu Rajakumar: Sure. With all of this happening, conspiracy theories are abound. Tell us about some of those conspiracy theories that you heard about and that you had to weigh up as you were digesting all that information.

Jamie Zakalik: There are a lot of conspiracy theories around this whole DeepSeek success and the alleged cost of compute that they use to train their model. I'll run through a couple of these. I do want to say that it seems like, from what I've heard, we can take most of what DeepSeek said at face value, but that doesn't mean that the market extrapolated it in the correct way. Let's run through some of those conspiracy theories because it's fun.

Anu Rajakumar: [laughs] Yes, absolutely.

Jamie Zakalik: First, I think it's worth pointing out that the $5 million to $6 million that they quoted, that was really the cost for the final training run. During an AI model training process, these AI labs often try different strategies and do a bunch of different things to train the best model possible, and that trial and error has costs associated with it, so that was not included in the $5 million to $6 million, and it could have been much larger. I've seen some estimates of $100 million.

The second thing is there were a lot of rumors that DeepSeek actually had access to tens of thousands of NVIDIA GPUs, not the roughly 2,000 they said they used, and had access to things like the flagship H100 chips that I talked about before, but they are not allowed to have access to these chips because they are on export control, so they couldn't disclose it, and so the cost could actually be a lot higher than they claimed.

The last one I'll point out is there were a lot of questions around what the R1 model was developed off of. The DeepSeek model apparently answered questions with significant similarity to OpenAI's models, and sometimes even referred to itself as ChatGPT. People were questioning whether the model was built off the shoulders of a US model, which cost hundreds of millions or billions to train, or, honestly, more likely if it used synthetic data from a model like OpenAI's o1 to train its own model, which is like having the answers to the test before taking the test. It just makes it a lot easier, and that would not, per se, be allowed, but apparently, a lot of AI labs do this in practice because it helps with the training process.

Anu Rajakumar: Great note. Very helpful to read between the lines of some of those juicy conspiracy theories. Jamie, just bringing this discussion together now, tell us what have been the actual implications of these DeepSeek revelations. When you look at companies, have they been reassessing their CapEx spending plans because of all this information that's come out?

Jamie Zakalik: Yes. The initial concern is, if it costs one-twentieth of what we thought to train a model, should the CapEx expectations and NVIDIA's potential revenue be one-twentieth of what we thought before? What we've seen in practice is that hasn't been the case at all. In fact, DeepSeek came out just before the Q4 earnings season kicked off, and we heard updated CapEx forecasts from all the top AI hyper-scale spenders, and what they said is they're actually spending more. If you look at CapEx expectations from just before DeepSeek came out to today, the cumulative expectation is up 10% or 15%.

Why might that be? One reason is pre-training. That $5 million to $6 million cost for pre-training a model is really only one part of the equation today. NVIDIA reported earnings last night, where we're recording on the Thursday after NVIDIA's earnings, and they talked about how there's multiple scaling laws happening now. Originally, models were trained almost exclusively before their use. You inputted parameters or data, and you train the model, and then you had a finished model that you could use. Today, AI developers have found new ways to make the models even better.

One of those is post-training. After the model is trained, once it's being used, you have what's called reinforcement learning, either with human, or AI feedback, and this helps the model continue to improve. This post-training is more compute-intensive than the pre-training. Then on top of that, the inference piece is also no longer just simply answer retrieval, but these models are incorporating reasoning, which is the model actually thinking and learning during the inference phase. NVIDIA said last night that this too is 100 times, and in the future could be 1,000 times more compute-intensive than before. All these point to the need for more compute and compute needs continuing to grow.

The other piece is this concept called Jevons paradox, which is basically the idea that increasing the efficiency and lowering the cost of a resource actually leads to an increase, not a decrease in total demand for that resource. Basically, lower cost encourages greater use of the resource. We've seen this in practice, an influx of demand for the lower-cost DeepSeek model after its release. I also see this in practice all the time with NVIDIA's chip roadmap. Every year, NVIDIA comes out with a new chip that can hypothetically double the performance at the same power or keep the same amount of performance at half the power.

Power is a cost. None of their customers ever reduce the amount of power they use. They just do more with their existing footprint and actually increase power and do even more. That's what we've seen and why these CapEx plans are still on track.

Anu Rajakumar: Yes, that's really helpful to hear, Jamie. I wanted to just highlight something that you said. You've mentioned this a couple of times, just the CapEx spending, you mentioned the top four US hyperscalers, north of $300 billion, which is a massive increase in just a couple of years. Just before this, you said the expectation of CapEx spending is up. I'm going to ask you maybe the hardest and unanswerable question, but I think it's the one that we're all wrapping our heads around, and that is about how we should be thinking about ROI, Return on Investment, in AI whether you are the company spending the money or whether you're the investor investing in these companies.

We're talking about tens to hundreds of billions of dollars being spent by the biggest companies in the world on AI. While the same time the cost of AI in some ways appears to be falling. How do we reconcile that? How should we be thinking about this from an ROI perspective?

Jamie Zakalik: Oh, I was really hoping you wouldn't ask me that question because that is really the $1 trillion question or $10 trillion question. That is the most important question because, at the end of the day, generative AI is cool. It's fun to send a poem to your friends and family that's customized about your experience, but if it doesn't have real-world quantifiable, tangible benefits, it's not really necessary. That question about ROI comes up a lot.

We talked about 300 billion being spent by these large cloud players. There's two angles to that. They have their internal business that they're incorporating AI into or using AI to drive efficiencies. Then they have these external cloud businesses where they're renting out AI to every enterprise across the world. On the internal side, I can give examples. It's hard to measure ROI on a broader basis right now, but if you look at Meta, they're one of these big AI spenders.

Their business model is serving content and serving ads to users. If they can figure out a way to make that content more relevant and more engaging and keep people on longer, and also feed ads that are extremely relevant and also customized for each individual user, they can drive a lot more revenue and user engagement, which is very important to their business model.

On the external front, AI across enterprise, I think this will take time, and I think that's where the ROI piece, there can be a mismatch between spend and when the ROI is actually achieved, but I think there are so many use cases in terms of driving productivity improvements but also new industries altogether that can come about from this technology. If you think about new industries, I think about markets like the education market. Everyone can have a personalized tutor that knows everything, can explain it to you with real-world examples and at different levels of complexity based on your knowledge base.

You think about things like customer service, this is a huge expense for every enterprise. Imagine if the chatbot that you talk to actually has human-like abilities. You really could reserve your need to talk to a human for a select number of use cases and that reduces wait times, it improves customer satisfaction. Things like customer satisfaction are really hard to measure. Does that keep a customer? If you really like Delta's customer service and you prefer to pay an extra $100 to fly Delta, how do they measure that? The ROI argument it's legitimate. It is really hard to measure, but there are ways that this technology can drive incremental value to enterprises, both the ones spending and the ones that are using some of their capacity.

Anu Rajakumar: Now, Jamie, turning to geopolitics, there appear to be growing headwinds for semiconductors related to export bans. How are these developments shaping the industry, and what should investors be watching for?

Jamie Zakalik: Export restrictions have become extremely relevant for the semiconductor industry overall. Maybe it's helpful to give some just brief background of why that is. If we rewind to the 2014-2015 timeframe, China is a massive consumer of semiconductors. I don't think people realize this, but that region consumes something like a third of all chips. In the 2015 timeframe, they produced almost none of them themselves, so those are all being imported, and it's a critical import for their economy.

The Chinese government launched this initiative called Made in China 2025 and the goal of that was to produce 70% of what they consume internally by 2025. For the US, this Made in China initiative was just another prime example of the Chinese government promoting unfair business practices to disadvantage external and especially US businesses. As a response, we've been in this seven-year semiconductor trade war. First with the government banning shipment of any US chips, and they actually use what's called the Foreign Direct Product Rule to ban non-US chips that use US technology to companies like Huawei, which is a Chinese tech giant.

More recently, they've had more targeted export bans on specific leading-edge semiconductor equipment that are used to manufacture semiconductors, and then the most advanced AI chips. China's restricted on what they can get from NVIDIA. Initially, the US restricted NVIDIA's ability to ship their flagship H100, which I mentioned, and NVIDIA was able to make a toned-down H800, which is what DeepSeek used to train on.

Since then, those ships have also been banned, and NVIDIA is only now able to ship an even more toned-down chip called the H20. Just to put that in perspective, I've seen estimates that H20 has around a seventh of the computing power of H100. NVIDIA's latest chip, the GB200 has about 30 times the computing power of H100. One-seventh, 30, we're talking about 200 times the performance delta between what China can get right now and what the US is able to deliver.

Anu Rajakumar: Wow. All right. Those are helpful to put into perspective. Jamie, just looking ahead, beyond the usual suspects like AI and data centers, what are some of the other exciting growth areas in semiconductors? Just reference, I'm thinking physical AI and robots, all that stuff. What are the long-term implications for semiconductors that you see?

Jamie Zakalik: Thank you for that question because semis are more than AI, they are more than this data center build that we're seeing right now. Obviously, AI has become a huge focal point because it is a market that's gone from not zero, but virtually zero to one of the largest semiconductor markets in the matter of a couple of years. I don't want to discount that, but even within AI, you touched on robotics and physical AI, there are areas that are still in their infancy and get drowned out by this singular focus on this data center scale training AI investment. I'm a big believer in the long-term diversified attractiveness of the semiconductor market.

At a high level, it's because anything that is smart and connected needs semiconductors. I think automotive is a really great example. You think about the car that you grew up with, maybe you had a CD player, maybe, if it was--

Anu Rajakumar: Cassette player, but yes.

Jamie Zakalik: Yes. Exactly. Cassette player. Now think about the newest car you've been in recently. It's night and day in terms of driver, passenger experience, safety features, all these different types of improvements. When you think about semiconductor content in a car, a few years ago, it was maybe $300 or $400. Today the average is maybe north of $600 or so, but if you look at the highest-end vehicles, they can have north of $2,000 of semiconductor content. I think that's a trend that continues.

You see that same content trend in other markets like healthcare, aerospace, and defense, energy infrastructure, smart cities, physical AI, robotics. These things, again, robotics, I think it'll take time, but I think there will be a lot more robots in the world in a few years than there are today. I think there's a lot of exciting opportunities for semis broadly outside of AI.

Anu Rajakumar: Great. Thanks for that. I could see your excitement, like you lit up when you're like, "There's other stuff as well other than ChatGPT." Thank you for sharing that. Jamie, before you go, I do have one bonus question for you. Let's go back to your childhood. When you were a kid, what was something that you encountered or experienced, which at the time felt like it was some mind-blowing, futuristic technology that, I don't know, you couldn't even imagine would be in the future?

Jamie Zakalik: It's super relevant because AI a few years ago, I think people would've said that about, and now it's here. Actually, I always think back to when I would go to Disney World as a kid and there was this ride called Spaceship Earth and the giant golf ball at EPCOT. They had this scene where someone was video chatting with someone in China. Today you would just say, oh, Zoom. At the time I said, oh, that's impossible. You can't see and talk to someone on a screen that far away. That's nuts. I really thought that was something that could never happen and now it is something we do every day in our job.

Anu Rajakumar: You don't even think twice about it.

Jamie Zakalik: You don't even don't twice about it. I'm sure that AI and some of the stuff we talked about today, which seems really, really futuristic, like robotics maybe in a few years we were saying that was something that at the time I thought was crazy and now it's part of our everyday life.

Anu Rajakumar: Absolutely, so true. Today's discussion focused on the transformative role AI is playing across industries, and we've covered a lot of ground. Jamie, you've explained the broader context of the constant innovation, AI, the accelerating advancements in cost efficiencies. We talked about NVIDIA and its long-term strategy. We talked about how investors can navigate the volatility really inherent in this rapidly evolving space. We touched on geopolitics and finally the outlook for growth areas in semiconductors beyond AI. Jamie, thank you very much for sharing your expertise and helping us better understand this pivotal moment in the AI-driven transformation of our world. It's been a pleasure having you on the show.

Jamie Zakalik: Thanks so much for having me. This was fun.

Anu Rajakumar: To our listeners, if you've enjoyed what you've heard today on Disruptive Forces, you can subscribe to the show from wherever you listen to your podcasts, or you can visit our website at nb.com/disruptiveforces, where you'll find previous episodes, including one from August 2024, where I spoke with Eileen Furukawa from Neuberger Berman's Emerging Markets Equity team, where she spoke about the global tech sector and in particular the semiconductor industry. That episode's titled Let the Microchips Fall Where They May. Please check it out. As always, thank you for listening.

Announcer: This podcast contains general market commentary, general investment education, and general information about Neuberger Berman. It is for informational purposes only, and nothing herein constitutes investment, legal, accounting, or tax advice or recommendation to buy, sell, or hold a security. This communication is not directed at any investor or category of investors and should not be regarded as investment advice or suggestion to engage in or refrain from any investment-related course of action.

All information is current as of the date of recording, and is subject to change without notice. Any views or opinions expressed may not reflect those of the firm as a whole. This material may include estimates, outlooks, projections, and other forward-looking statements. Due to a variety of factors, actual events or market behavior may differ significantly from any views expressed. Please refer to the disclosures contained in the episode notes and episode details, which are an important part of this communication. Investing entails risks, including the possible loss of principle. Past performance is no guarantee of future results.