The AI chip arena is witnessing intensifying competition, with a new production-ready startup claiming its product outperforms the world's highest-valued company, NVIDIA.
Headquartered in Silicon Valley, just three miles from NVIDIA's base, D-Matrix asserts that its new chip, named Corsair, can run inference tasks up to ten times faster than NVIDIA's discrete GPUs while consuming only one-fifth of the power, though this advantage is specific to smaller-scale computational workloads.
This new inference chip employs an innovative memory architecture, a design philosophy shared by other challengers like Cerebras and Groq. The surging demand for computing power from major tech firms underscores a clear industry trend: ample room remains for smaller, specialized chipmakers to carve out niche opportunities.
Cerebras, founded in 2015, completed a major IPO last month, raising over $55 billion and achieving a valuation exceeding $500 billion. Groq was acquired by NVIDIA for $20 billion last December, marking the AI giant's largest-ever acquisition. NVIDIA subsequently unveiled a new Language Processing Unit based on Groq's technology at its March GTC conference.
In an interview, D-Matrix co-founder and CEO Sid Sheth stated, "A trillion-dollar market is forming. I have absolutely no intention of selling the company. This market is more than capable of supporting another public chip company."
Founded in 2019, D-Matrix has raised approximately $500 million, with a post-money valuation around $2 billion. Notably, Microsoft's venture arm, M12, is among its investors. This is significant as Microsoft itself harbors substantial chip ambitions, developing its own Maia 200 AI inference chip, co-creating a new PC processor with NVIDIA, and recently announcing a proprietary quantum computing chip.
Sheth has not disclosed Corsair's specific client list but revealed securing purchase intentions from major cloud providers, new cloud players, and cutting-edge AI labs, all actively expanding their computing capacity. D-Matrix began bulk shipments to customers this month, with about 90% located in the US and overseas buyers concentrated in the Middle East and Southeast.
Bernstein semiconductor analyst Stacy Rasgon commented, "Most customers will likely use D-Matrix chips alongside NVIDIA's, as different chips excel at different types of compute tasks. It appears this company has secured a significant number of real customer engagements."
D-Matrix's Corsair chip integrates memory and compute units into a single die, enabling low-power, low-latency inference. Like Groq and Cerebras, D-Matrix utilizes Static Random-Access Memory (SRAM), which can be manufactured and integrated on-chip by logic foundries like TSMC. Traditional GPUs rely on vast amounts of Dynamic Random-Access Memory (DRAM), which is packaged separately as High Bandwidth Memory and stacked around the logic chip.
Sheth noted, "Our product won't face DRAM supply bottlenecks because the entire solution's performance doesn't depend on DRAM."
Rick Bahr, a consulting professor of electrical engineering at Stanford University, pointed out a significant limitation of the D-Matrix approach: SRAM architecture cannot support inference models with ultra-large parameters. While on-chip SRAM offers extremely short data transfer distances and impressive inference speeds, it cannot accommodate the trillion-parameter models used by leaders like OpenAI and Anthropic.
Sheth clarified that Corsair is designed specifically for AI inference scenarios, prioritizing interactive response speed over hosting massive language models, making it suitable for chatbots, voice assistants, and intelligent agent tools.
Citing test data from Gimlet Labs, D-Matrix claims that when deployed alongside NVIDIA's Blackwell GPU, Corsair can achieve inference speeds ten times faster than a standalone GPU, reduce total cost of ownership to one-third, and improve energy efficiency by up to five times.
Last week, NVIDIA CEO Jensen Huang asserted that with its flagship Vera Rubin system, NVIDIA remains the leader in cost-effective inference, emphasizing that performance is not the sole criterion.
Sheth explained that D-Matrix packages four Corsair chips into an accelerator card that plugs directly into a data center server rack slot, with each card priced in the tens of thousands of dollars. He stated this plug-and-play deployment is a core advantage over Cerebras and Groq, calling Corsair "the densest SRAM solution on the market," with a single server supporting up to 128GB of SRAM.
D-Matrix has also partnered with Arista, Broadcom, and Super Micro to launch a rack-scale system, SquadRack, specifically designed for deploying its chips in AI data centers.
The Corsair chip is manufactured using TSMC's 6nm process. The next-generation product, codenamed Raptor, is planned for release next year using TSMC's 4nm process, with Sheth suggesting production could be allocated to TSMC's Arizona facility in the US.
Sheth concluded, "Building complete computing solutions tailored for AI inference will be the industry's most significant opportunity."
Comments