By Eric J. Savitz
One of the biggest threats to Nvidia's dominance of the AI chip market will come from some of the company's biggest customers.
Amazon, Alphabet, Meta, and Microsoft are four of the largest players in AI cloud computing. They also happen to be among the biggest buyers of Nvidia's graphics processing units, or GPUs, which have become integral in training AI software and running inference applications off AI models.
The other thing the cloud giants have in common is they're all building chips of their own to supplement their Nvidia supply.
While none of the Cloud Four are likely to sell chips to companies like Dell, Super Micro, or other hardware providers, their chips could still make a dent in Nvidia's 90%-plus share of the AI GPU market.
Here's why: All four of the cloud providers want and need to take control of surging capital spending budgets. Meta, Microsoft, Amazon, and Google-parent Alphabet are spending a fortune on AI infrastructure. They're combined capital spending for fiscal 2024 is expected to reach $178 billion, according to FactSet estimates, up more than 26% from last year.
Much of their cost control will come by designing their own "stack," including software, hardware and chips -- not unlike Apple's approach.
Amazon jumped into the chip business in 2015 via the acquisition of Israeli chip-design firm Annapurna Labs. That deal has resulted in three chips: a CPU called Graviton, which Amazon says offers 40% better price performance than comparable x86 chips; Trainium, for training large models; and Inferentia, for AI inference workloads.
Gadi Hutt, director of business development for Annapurna, said Amazon started looking at the AI chip market in 2016. "There was no Gen AI, and there were no LLMs [or large language models], but machine learning was growing as a use case at Amazon and with our customers," he said. "They all came to us with basically the same product statement, which was that we want to do more, but it's too expensive." Solving that problem underlies Amazon's AI approach.
"Compared to other solutions that are available in the cloud, mainly Nvidia, you will save money and you'll have high performance, in some cases even higher," Hutt says.
Amazon is recruiting some of the biggest AI spenders, including Anthropic, the OpenAI rival, which uses Trainium to train its AI models. Airbnb, ByteDance, Snap and Deutsche Telekom are Inferentia customers. Amazon uses its own chips to run Alexa, Amazon Ads, and its new Rufus shopping bot.
Like Amazon, Alphabet's Google has been building AI chips for nearly a decade, and like Amazon, it sees an advantage in creating the full system -- not just chips, but also the related software, networking infrastructure, and storage. Google calls its chips "tensor processing units," or TPUs. ("Tensor" is a reference to matrix multiplication, the math behind AI software.) Google is on its fifth-generation TPU, which comes in two varieties, one focused on performance -- to create models at maximum speed -- and the other tuned for efficiency.
Google uses TPUs to build large language models -- including its widely deployed Gemini software. It also relies on TPUs to drive Google Mail and other services. Among the many start-ups that rely on Google TPUs are Character AI, which makes personalized chatbots, and Midjourney, which provides text-to-image software.
Mark Lohmeyer, vice president of compute and machine learning infrastructure at Google Cloud, says the company started the project over a decade ago, when a few engineers floated a thought experiment -- how much compute it would take if Google users interacted a few minutes a day with search through voice prompts? He says the answer was that the amount of required general purpose computing it would have required was astounding -- spurring the work on processors or AI and machine learning.
Earlier this month, Meta launched its second-generation AI chip, dubbed MTIA, or Meta Training and Inference Accelerator. The parent of Facebook and Instagram is using the MTIA chip -- again, produced by TSMC -- to power social media rankings and ad models.
"MTIA is a long-term venture to provide the most efficient architecture for Meta's unique workloads," the company said in a blog post announcing the new chip. "As AI workloads become increasingly important to our products and services, this efficiency will improve our ability to provide the best experiences for our users around the world."
Microsoft is the furthest behind among the cloud players when it comes to chips, but it has entered the race. Last year, the company unveiled the Azure Maia 100 AI Accelerator, a chip for model training and inferencing. So far, Microsoft has limited use to internal workloads, like Github Copilot, Bing, and OpenAI's GPT 3.5, but access for customers is coming.
"We are optimizing and integrating every layer of the stack," says Rani Borkar, corporate vice president for Microsoft Azure hardware systems and infrastructure. "It's not just about silicon. We are reimagining every layer, from silicon to servers to system to data center infrastructure, to maximize performance and efficiency."
Borkar adds that Microsoft wants to give cloud customers "optionality." Says Borkar: "It's not this or that, it's this and that. ... we need a diversity of suppliers."
Write to Eric J. Savitz at eric.savitz@barrons.com
This content was created by Barron's, which is operated by Dow Jones & Co. Barron's is published independently from Dow Jones Newswires and The Wall Street Journal.
(END) Dow Jones Newswires
April 19, 2024 01:30 ET (05:30 GMT)
Copyright (c) 2024 Dow Jones & Company, Inc.
Comments