Artificial intelligence has delivered seemingly daily wonders for the past 18 months. For investors, the biggest surprise has been the rise of Nvidia, which has come from humble roots to thoroughly dominate the market for AI-related chips.
Once known mostly for building PC add-on graphics cards for gamers, Nvidia has transformed its graphics processing units, or GPUs, into the beating heart of the AI revolution, powering the creation of large language models and running the inference software that leverages them in data centers around the world. Nvidia has been nearly alone on the field, with more than 90% market share.
But fresh competition is coming—from companies big and small—and the battle will be fierce. The stakes couldn’t be bigger: Lisa Su, the CEO of Advanced Micro Devices, has sized the AI chip market at $400 billion by 2027. Intel CEO Pat Gelsinger has projected a $1 trillion opportunity by 2030. That’s almost twice the size of the entire chip industry in 2023.
Nvidia’s Jensen Huang has built a company that is universally respected and admired, but chip buyers aren’t keen on relying on a single source. Hardware companies such as Dell Technologies, Hewlett Packard Enterprise, Lenovo, and Super Micro Computer can’t get enough Nvidia chips to meet customer demand—and they’d like alternatives. Cloud providers like Amazon.com and Alphabet’s Google want more options so badly that they are designing their own chips. And companies that rely on AI-based systems want more computing resources at more manageable costs than they can get now.
Nvidia’s success is now an opportunity for everyone else.
It’s hard to find a product of any variety that has had more impact on the financial markets so quickly than the Nvidia H100 GPU, which launched in March 2022.
Nvidia’s share price has more than tripled since the H100’s debut, boosting the company’s market value to $2.1 trillion. Among U.S.-listed companies, only Microsoft and Apple have higher market caps. And no other chip company is anywhere close.
This is no GameStop or Trump Media & Technology. In fact, Nvidia is the anti-meme stock: The company’s revenue growth has actually outpaced the stock gains. For its fiscal fourth quarter ended on Jan. 28, Nvidia posted revenue of $22.1 billion, up 265% from a year earlier. The company’s data center revenue was up 409%.
A few weeks ago, Nvidia launched its latest marvel, the Blackwell B200 GPU, which CEO Huang says dramatically outperforms the H100. With Blackwell, Nvidia raises the bar for its rivals. Nvidia for the foreseeable future will sell as many Blackwells as it can make—or, to be more precise, that partner Taiwan Semiconductor Manufacturing can make for it.
Huang has said Blackwell GPUs will cost $30,000 to $40,000 apiece. The current H100s sell in the same range. But the chip prices aren’t the whole story. AI customers want to run workloads in the shortest time, at the lowest cost, with the highest accuracy and reliability, drawing as little power as possible. There are a number of companies that think they can do that as well—or better—than Nvidia.
Nvidia’s rivals fall into three groups: big chip makers, cloud computing vendors, and venture-backed start-ups. With a $1 trillion market at stake, this won’t be winner-take-all. It isn’t game over. It’s game on.
Nvidia’s most obvious challengers are Advanced Micro Devices and Intel.
AMD shares have rallied 71% over the past 12 months, aided by the market’s perception that its new MI300 GPUs will chip away at Nvidia’s stranglehold on the market. That hope is inspired by AMD’s success at stealing market share from Intel in PC and servers.
“AMD is really the only other company on the field,” contends Andrew Dieckmann, general manager of AMD’s data center GPU business. “We’re the only other solution being adopted at scale within the industry.” He says that AMD chips outperform Nvidia’s H100 for many inference workloads, while offering parity for model training. But AMD’s other asset is that it isn’t Nvidia.
“For the very large users, they are not going to bet their entire franchise on one supplier,” Dieckmann says. “There is an extreme desire for market alternatives.”
AMD CEO Lisa Su said on the company’s most recent earnings call that she now expects 2024 GPU revenue of $3.5 billion, up from a forecast of $2 billion a quarter earlier.
Intel is coming from behind—and the stock has struggled for years—but the company opened eyes this month with the launch of Gaudi 3, its third-generation AI accelerator chips for training and inference. Intel contends that Gaudi 3 is faster than Nvidia’s H100 for both AI tasks, while using less power—and that Gaudi 3 will be competitive with Blackwell.
Intel, which is now spending billions to build out chip fabs in Arizona and Ohio, should be advantaged over the long run by having its own source of supply in a market with a severe supply shortage. But not yet: Gaudi 3 will be produced by TSMC, just like AI chips from Nvidia and AMD.
Jeni Barovian, vice president of Intel’s Network & Edge group, credits Nvidia for “establishing a foundation for this revolution,” but says Intel doesn’t intend to be left behind. Customers want alternatives, she says. “They don’t feel like they’re getting a choice today.”
Then there’s Qualcomm, another long-term leader in the semiconductor arena. The mobile-phone chip company has taken technologies originally designed for smartphones and applied them to the cloud in an AI inference chip it calls the Cloud AI 100. Qualcomm ultimately seems more interested in the opportunity to serve the edge of the networks, on laptops and phones.
Qualcomm senior vice president Ziad Asghar thinks that, over time, more inference workloads will be handled on “edge devices.” The theory is that it’s cheaper, and more data-safe, to stay out of the cloud. “The center of gravity in inference is shifting from the cloud to the edge,” Asghar says.
Less visible but no less serious about competing in AI chips are the internal teams at four cloud-computing giants—Amazon, Alphabet, Meta Platforms, and Microsoft. All four are designing proprietary chips for both their own internal needs and to serve cloud customers. The competitive story here is less direct—none of the cloud leaders sell chips to third parties.
Nonetheless, they’re still a threat to Nvidia. Meta, Microsoft, Amazon, and Google together are spending a fortune on AI infrastructure. Those four are expected to have combined fiscal-2024 capital spending of $178 billion, up more than 26% from a year earlier. Microsoft alone will see capex increase 53% this year, according to estimates tracked by FactSet. Spending will jump 31% at Alphabet, and 26% at Meta. All four are building chips in part to gain better control of their spending—and they all say they can get there by controlling more of the “stack,” including software, hardware, and chip design—not unlike Apple’s approach to hardware design.
Nvidia declined to comment for this article, noting that the company is in its pre-earnings quiet period. But Huang did address the question of competition in the recent past.
“We have more competition than anyone on the planet,” Huang said at a March event at the Stanford Institute for Economic Policy. “Not only do we have competition from our competitors, we have competition from our customers.” His view is that the high levels of integration and efficiency built into Nvidia’s systems make it tough for rivals to keep up. Huang has said that the company’s total cost of operation is so good that “even when the competitors’ chips are free, it’s not cheap enough.”
That hasn’t stopped Big Tech from trying, and they’re being joined by a host of venture-backed start-ups. Some of those start-ups are hoping to sell chips and systems to server and cloud providers, but most of them are trying to disrupt the market by offering cloud-based services directly to customers.
The most intriguing of the group might be Cerebras Systems, which reportedly is planning a 2024 IPO.
Chip Choices
Nvidia has dominated the market for AI chips so far, but the list of competitors is growing. A look at some of the contestants.
In March, Cerebras unveiled its largest chip to date, the Wafer Scale Engine 3, or WSE-3. At 72 square inches, it’s the largest commercial chip ever made. The H100, by contrast, is roughly one square inch. Rather than trying to network lots of chips together—an engineering challenge—Cerebras is simply packing all of the power onto a gargantuan semiconductor.
Cerebras’ chip has four trillion transistors, 50 times the computing power of the H100.
Cerebras is bundling the chips into a computing platform called CS-3, which it says can train a large language model like Meta’s Llama in one day, versus one month for Nvidia-based platforms. Cerebras doesn’t sell chips directly. While it’s willing to sell full hardware systems, most of the start-up’s revenue comes from selling access to its systems the same way cloud vendors do. Among its customers: the Mayo Clinic, GSK, and the Lawrence Livermore National Laboratory.
Cerebras CEO Andrew Feldman says the company built eight times as many systems in 2023 as it did the prior year, and it expects the total to increase 10 times in 2024. Cerebras had 2023 revenue of $79 million, and has reached cash flow break-even. The company has raised $715 million in venture capital, and was valued at $4 billion in its most recent round, in 2021.
“The opportunity is to bring supercomputer performance to large enterprises without supercomputer overhead,” Feldman says.
The secret to that, he says, is the dinner-plate-size chips. “Big chips means you don’t have to break up work.”
Jonathan Ross, who played an important role in developing Google’s AI chips, is now running an AI chip start-up called Groq—not to be confused with Elon Musk’s Grok AI model. (All of this grokking stems from a word coined by Robert Heinlein in his 1961 novel, Stranger in a Strange Land; it means to fully understand something in the deepest possible way.)
Like Cerebras, Groq’s strategy is to sell compute time on a consumption basis, rather than selling chips to hardware and cloud companies. Ross says Groq can run popular models like Meta’s Llama 2-70b at 10 times the speed of Nvidia-based systems. (You can try it free on the company’s website.) After Nvidia’s Huang unveiled Blackwell a few weeks ago, Groq cheekily issued a press release that simply said, “Still faster.” The company has announced $367 million in funding to date; Ross says it’s raised additional capital that hasn’t yet been announced.
AI chip start-up d-Matrix is focused on AI inference applications in the data center, leaving the model-building to Nvidia and others. Founded in 2019, d-Matrix uses a design called “in-memory compute,” which CEO Sid Sheth says is an idea that has been around for 30 or 40 years—an approach to speed up computation—but which never had a really good application until AI emerged. D-Matrix, which has raised $160 million, expects to start selling chips next year.
SambaNova, which has raised $1.1 billion from venture investors, is another company taking a systems-based approach to serving the AI market. In addition to chips, its approach aggregates 54 open-source AI models, including those from Google and Meta.
One surprising element of the SambaNova story is that, for many customers, it is installing physical systems in their data centers. “The large majority of enterprise data still sits on premises,” says CEO Rodrigo Liang. He notes that customers come from areas such as banking, healthcare, and government, including the national laboratories at Sandia, Lawrence Livermore, and Los Alamos.
There are plenty of other AI chip start-ups out there. Rain AI, which has seed funding from OpenAI CEO Sam Altman, is focused specifically on energy efficiency, and running large language models on edge devices. Lightmatter is using photonics—light-based technology originally built for quantum computing applications—to improve the speed of networking in AI data centers.
To be sure, these start-ups have big ideas, but they’re still operating at a small scale. It could be years before they’re ready to take on Nvidia. There are potentially big wallets ready to accelerate their progress, though.
Bloomberg has reported that SoftBank Group is considering a $100 billion investment to fund a new AI chip company. (SoftBank is also a SambaNova investor and customer.) Meanwhile, OpenAI’s Altman, according to The Wall Street Journal, has penciled in a plan to invest up to $7 trillion—a figure that seems implausibly large—to build dozens of new chip fabs. Neither SoftBank nor OpenAI would comment on those reports.
The emergence of the growing number of new and potential competition isn’t likely to end Nvidia’s reign as the AI chip champion. But it does mean that Nvidia’s unobstructed domination of a $1 trillion market is coming to an end. Starting now.
