Alphabet's Google is advancing a new initiative to optimize its AI chips for PyTorch, the world's most widely used AI software framework, in a bid to challenge NVIDIA's long-standing dominance in the AI computing market, according to sources familiar with the matter.
This initiative is part of Google's ambitious strategy to position its Tensor Processing Units (TPUs) as a viable alternative to NVIDIA's market-leading GPUs. As Google seeks to demonstrate returns on its AI investments to shareholders, TPU sales have become a key growth driver for its cloud business revenue.
However, sources indicate that hardware alone isn't enough to drive widespread adoption. The internal project, codenamed "TorchTPU," aims to eliminate key adoption barriers by ensuring full compatibility between TPUs and PyTorch while improving developer accessibility. This would address the needs of clients already using PyTorch-based architectures. Google is also considering open-sourcing certain components to accelerate adoption.
Compared to previous attempts to support PyTorch on TPUs, Google has allocated significantly more organizational focus, resources, and strategic priority to TorchTPU. This shift comes as potential TPU adopters increasingly cite software ecosystems as implementation bottlenecks.
PyTorch, an open-source project heavily backed by Meta Platforms, is among the most popular tools for building AI models. In Silicon Valley, few developers write code line-by-line for specific chips from NVIDIA, AMD, or Google. Instead, they rely on tools like PyTorch—a collection of pre-built code libraries and frameworks that automate common AI development tasks. First released in 2016, PyTorch's evolution has been closely tied to NVIDIA's CUDA software, which some Wall Street analysts view as NVIDIA's strongest competitive moat.
NVIDIA engineers have spent years ensuring PyTorch-based software runs optimally on their chips. In contrast, Google has long directed its vast internal software teams to use Jax, another framework, with its TPUs optimized via the XLA tool. This focus on Jax has created a significant gap between Google's chip optimization approach and mainstream developer preferences.
A Google Cloud spokesperson declined to comment on project specifics but confirmed to Reuters that the move would expand customer choice: "We're seeing surging demand for both TPU and GPU infrastructure, with accelerating growth. Our priority is delivering the flexibility and scale developers need, regardless of their hardware choice."
Historically, Alphabet reserved most TPU capacity for internal use. This changed in 2022 when Google Cloud gained control over TPU sales, significantly increasing its allocation. As AI interest grows, Google is scaling production and external sales to capture market opportunities.
However, PyTorch—used by most global AI developers—remains incompatible with Google's Jax-optimized chips. This forces developers to undertake costly engineering work to match NVIDIA's performance—a significant hurdle in fast-moving AI development.
If successful, TorchTPU could dramatically reduce migration costs for firms seeking NVIDIA GPU alternatives. NVIDIA's dominance stems not just from hardware but also its CUDA ecosystem, deeply embedded in PyTorch as the default solution for training and running large AI models.
Enterprise clients have reportedly told Google that TPUs are difficult to deploy for AI workloads because they traditionally require switching to Jax instead of the widely adopted PyTorch.
To accelerate development, Google is collaborating closely with Meta Platforms, PyTorch's primary developer. Earlier reports indicated the two tech giants were negotiating a deal that could grant Meta expanded TPU access.
Initially, Google offered Meta a fully managed service: deploying Google-designed chips running Google software, with Google handling operations. Sources say Meta has strategic incentives to support TPU software adaptation—it could lower inference costs and reduce reliance on NVIDIA GPUs, strengthening its supply chain leverage.
Meta declined to comment.
This year, Google began selling TPUs directly to client data centers, no longer restricting access to its cloud platform. Recently, Google veteran Amin Vahdat was appointed to lead AI infrastructure, reporting directly to CEO Sundar Pichai.
Google's infrastructure serves dual purposes: powering its own AI products like Gemini and AI-driven search, while providing computing resources for Google Cloud clients, including Anthropic.
Comments