The ongoing DRAM supply crunch has grown so severe that major AI players are now resorting to stripping down decommissioned servers to recover usable memory modules. Nikhil Cherian, Google’s Senior Director of Supply Chain Infrastructure, recently noted that the AI sector has swiftly pivoted from being compute-constrained to memory-constrained, with high-performance memory now accounting for roughly 75% of the total bill of materials for a single AI server.
To break through this bottleneck, Google is attacking the problem on both software and hardware fronts. On the hardware side, the company has established an internal recycling supply chain that dismantles retired servers to harvest still-functional DDR4 memory modules. Cherian revealed that Google designed specialized hardware adapters to allow previous-generation DDR4 memory to connect to newer AI servers and is deliberately re-importing decommissioned machines to extract their DDR4 modules. This means that new AI servers, which would typically use DDR5, are in some cases being downgraded to run on salvaged DDR4.
On the software side, Google continues to refine underlying function libraries, model architectures, and KV Cache compression techniques, all aimed at reducing memory consumption per unit of compute. This year, Google introduced two TPU ASICs: the TPU8t, designed for training, and the TPU8i, geared toward inference. Each TPU8i chip is equipped with 288GB of HBM3e, delivering 8.6 TB/s of memory bandwidth, and integrates 384MB of on-chip SRAM to store active KV caches, minimizing the need to query external system memory. TPU8i servers use Google’s custom Arm-based Axion processors instead of traditional x86 host CPUs, relying on high-speed DDR5 memory architecture to handle host-level tasks like data preprocessing.
Yet, even with these advanced setups, Google is still compelled to substitute recycled DDR4 for DDR5 in certain scenarios, underscoring just how dire the current DRAM supply situation has become.
This article was originally published by 快科技. Reproduction requires explicit attribution to the source.
Comments