The artificial-intelligence revolution has entered a new phase, one in which running AI models, known as inference, is taking over as the main source of demand for AI computing. Nvidia was the winner of round one when training the AI models drove chip sales. But things change quickly in tech, and the company still has to convince the market and customers that it remains indispensable.
CEO Jensen Huang devoted his keynote address at Nvidia’s GTC conference this past week to make the case. He reminded everyone that Nvidia had spent two decades building an ecosystem of hardware and software that makes its platform the least costly for AI. By the end of his speech, Huang had delivered a vision of Nvidia that reminded me of just one other company: Apple.
For years, Wall Street didn’t appreciate that Apple was more than just a hardware firm. Apple’s version of consumer technology provides a carefully thought-out bundle. The hardware is expensive, but it comes with a lot of free software and services that bring everything together seamlessly. In the end, the platform is sticky and full of value.
This is sometimes called Apple’s “walled garden.” iPhones, Macs, and Watches work like one because Apple controls the entire technology stack: the chips, the devices, the operating systems, the applications, and the cloud services. It’s all developed together, so it all just works together.
You’re free to leave the garden through a well-hidden gate, but the flowers are nice and the sun is shining, so why would you?
Nvidia is employing that Apple model of full control in an entirely different market: AI computing. More and more, Nvidia is moving toward being a full platform with an ecosystem of hardware, software, and partnerships that could be sticky like Apple’s, notwithstanding growing competition in the AI chip market.
It begins with Nvidia controlling as many layers of data center infrastructure as it can, what CEO Jensen Huang calls “extreme codesign.” A lot of attention is paid to Nvidia GPU chips, the workhorses of AI data centers, but there are five other Nvidia chips inside its coming Vera Rubin AI server, each with a crucial role in making a product that can’t be matched. The chips work better because they are designed together to work together.
Nvidia also makes data center network switches that alleviate a key computing bottleneck. In the last quarter, networking sales were responsible for 16% of Nvidia revenue, up from 8% the year before. It’s now the fastest-growing unit in Nvidia’s reporting.
This year, Nvidia will integrate a new server design built around AI inference chips from start-up Groq. Vera Rubin will work in concert with Groq on demanding inference tasks. Creating a data center with mixed servers that collaborate with each other is a thorny problem that Nvidia solved with software called Dynamo. Nvidia’s hardware still leads the industry, but the deepest part of the company’s moat is all the software it’s created to run on its hardware.
Huang began his GTC keynote by talking about the 20th anniversary of Nvidia’s most important software known as CUDA, or Compute Unified Device Architecture. In 2004, Nvidia hired Ian Buck, an engineer fresh out of Stanford University, to create a way for programmers to use Nvidia GPUs for a lot more than just computer graphics and gaming. Two years later, CUDA was born.
Nvidia kept developing the software, and by 2012, AI researchers had made Nvidia’s platform their preferred kit. A whole generation of researchers grew up on it. When ChatGPT triggered the generative AI craze in 2022, no one was more prepared for it than Nvidia.
Buck remains a Nvidia employee.
Nvidia has continued to build the ecosystem on top of the GPU-CUDA combination. The company’s online code portfolio has 700 repositories, including specialized software for engineering, physics, weather, and medical science, along with tools for AI training, inference, and agents. These are active projects with new versions rolling out all the time. Over a third of the repositories have received updates in the past month.
Nvidia is also the world’s largest contributor to open-source AI models with 715 of them available for download. Over 90 of the models have been updated in the past month. Along with general language models, there are ones for math, science, robotics, and autonomous driving.
Nvidia has also built deep supply-chain relationships, including $95 billion in fiscal 2027 purchase agreements to make sure it gets to the front of the line at its most important vendors, which are seeing off-the-chart demand growth for high-end chips.
Nvidia’s commitments go downstream, as well, into its base of customers. In the last fiscal year, the company spent $17.5 billion to take equity stakes in AI start-ups, any of which could be the next Google or Facebook. Nvidia’s perpetual license of Groq technology cost another $20 billion.
Through it all, Nvidia has fostered a devoted set of customers, just like Apple. Samsung phones have long had better specs than iPhones but few iPhone users ever switch—all because that walled garden is a nice place to be.
Apple doesn’t just make iPhones, just like Nvidia doesn’t just make GPUs. Apple investors eventually figured that out. Before long, the market could have the same realization about Nvidia.
Comments