Good morning. It's been eight months since Nvidia announced its $20 billion deal with chip upstart Groq, which is focused on speeding up the use of AI models after they have been trained. The commercialization of that deal has started, and its success may tell us a lot about the extent to which AI agents in the enterprise live up to expectations.
What's a Groq? First, a bit of background. Nvidia's graphics processing units, or GPUs, have been the go-to chips for training large language models. Training has been the main AI application for the past few years. The focus is increasingly on the quickly growing realm known as inference, or the use of trained models to solve problems and get things done for users.
The Groq deal was designed to expand Nvidia's arsenal in inference computing. Last December, it licensed the company's technology and hired Groq founder and CEO Jonathan Ross and other members of the team.
Nvidia said Monday that its Groq 3 LPX was now in full production, with Nebius as the first AI cloud to adopt it.
It's timely given that AI agents demand ever-higher levels of inference as they run longer and focus on more complex tasks.
"Agentic AI creates two distinct computing challenges: efficiently processing enormous amounts of context and generating tokens with extremely low latency," Nvidia said.
The Groq chip complements Nvidia's Vera Rubin systems.
Nvidia said the Groq 3 LPX "is purpose-built to extend Vera Rubin's interactivity-the rate at which tokens are generated for an individual user, determining how quickly an agent can complete each step of its work." And faster generation, Nvidia said, enables agents to get more done while "maintaining a responsive user experience."
Nvidia isn't alone in this effort. As CNBC noted yesterday, "It's a competitive space. Smaller GPU maker Advanced Micro Devices announced earlier this year it would integrate its rack-scale systems with chips from Cerebras, which recently went public, focusing on low-latency inference."
The key is to build the infrastructure that supports enterprise-grade AI agents, or what Nvidia refers to as the responsive user experience. That rests on more than smart models and the software around them. It requires an increasingly capable foundation of hardware. It's coming to market now, and the next question is what companies will do with it.
Has your company's use of AI agents evolved in the past few months or has it been more or less static? Send your feedback to me at steven.rosenbush@wsj.com (if you're reading this in your inbox, you can just hit reply).
CEOs Want AI Agents to Read Your Messages
Executives are urging their staff to send messages in public Slack channels, open to anyone in the office to see, rather than privately pinging colleagues. They want AI agents crawling through messages to have unfettered access to every update on products, earnings and strategic initiatives, The Wall Street Journal reports.
Many leaders are examining how AI will change internal norms to deepen the technology's impact. Communication is one of them: Data on what people are writing and how they are working are becoming extremely valuable to companies.
On Our Radar
Oura-whose fitness-tracking devices are favorites of the finance set-is now weighing a September or October public offering after having filed its paperwork in May, The Wall Street Journal reports. The company is expected to fetch a valuation well above the $11 billion valuation from a funding round last year, and recently restructured its tech leadership. Elsewhere, Switch, which operates data centers, and SB Energy, which is backed by SoftBank and develops data centers and supplies power to them, are both meeting with investors in anticipation of offerings later this year.
Nvidia's current financial position looks rock solid to say the least. But the financial engineering it is using to keep revenue growing introduces risks that could eventually cause real pain, according to WSJ's Asa Fitch, writing in a Heard on the Street column. Nearly four years into an AI boom that has brought Nvidia hundreds of billions of dollars in profit, the company is increasingly tapping its financial strength to keep customers buying its chips.
The fight against Flock Safety is getting louder and more sophisticated, even as the company's AI-enabled cameras continue to spread across America. Flock has 120,000 cameras in 49 states that capture 20 billion license plates a month. The company says it is giving police a powerful tool to catch criminals. But the same bipartisan public anger that has risen up from towns and cities to stall data centers is now aimed at Flock's spreading camera network, WSJ reports.
One of Taiwan's biggest AI winners has nothing to do with chips-it's a furniture-component maker whose founder, Lin Tsung-chi, is now Taiwan's richest person. King Slide Works makes components like cabinet hinges and drawer slides. It also makes the rails used in AI server racks. That may sound mundane, but the rails support extremely heavy servers while keeping them cool, WSJ reports.
The WSJ Technology Council Summit
The WSJ Tech Council brings together CIOs, CTOs and CISOs advancing innovation and shaping the future. Join this trusted community where tech executives connect with peers to explore emerging trends and gain the perspective they need to stay ahead of disruption.
Request an Invitation
About Us
Follow Isabelle Bousquette on LinkedIn, Instagram, X, and TikTok for more behind the scenes on her tech and AI coverage, and lately, her contributions to the WSJ Leadership Institute's new Executive Resilience series, where she's profiling America's top execs about their fitness and wellness habits.
Follow Belle Lin on LinkedIn and X for her latest reporting on enterprise technology and AI.
Steven Rosenbush is chief of the enterprise technology bureau at the WSJ Leadership Institute. He also has a column. You can follow him on LinkedIn.
Tom Loftus is the editor of The Morning Download. He suggests following Isabelle, Belle and Steve on their various social channels. But if you insist, here's his LinkedIn.
Comments