Every layer of the AI infrastructure stack is simultaneously under pressure, presenting both opportunities and potential misjudgments.
In a recent podcast interview, Dylan Patel, founder of SemiAnalysis, systematically outlined the core dynamics and investment logic of the current AI infrastructure landscape.
His analysis covered model economics, the memory supercycle, the repricing of CPUs, the timeline risks for CPO, and structural opportunities in data center energy supply.
Anthropic has already achieved positive free cash flow, with the AI demand narrative beginning to materialize.
Addressing widespread market skepticism about AI return on investment (ROI), Dylan provided specific data points.
He revealed that Anthropic achieved positive free cash flow in the second quarter, is profitable, and has an annual recurring revenue run rate exceeding $50 billion with gross margins over 70%.
On the enterprise side, the productivity leap delivered by the latest AI models far outweighs the increase in compute costs, prompting companies to cut other software expenses to sustain their rapidly expanding AI budgets.
Memory faces a structural shortage, not a regular cycle.
Among all hardware categories, Dylan Patel's view on memory is the most resolute.
He stated this is not a short-term shortage but a structural one that will last for years, with potential for 2x to 3x further upside.
The core driver is the impact of reasoning models on KV cache requirements.
While traditional conversational inference had context lengths in the thousands of tokens, reasoning models like o1 have caused context lengths to explode, drastically increasing KV cache size and making memory the primary beneficiary.
Supply-side constraints will force the downstream market to reallocate limited memory resources, with price-inelastic consumer electronics likely bearing the initial brunt.
He predicts memory prices will continue to rise until AI secures the supply it needs, with long-term growth from trough to trough remaining undeniable.
CPU demand is driven by catch-up, with limited long-term extrapolation.
While CPUs have emerged as a new narrative in AI infrastructure this year, Dylan Patel offers a clear note of caution.
The recovery in CPU demand has clear logic: reinforcement learning requires significant CPU for environment validation, and agent-based reasoning relies heavily on CPU compute for tool calls and real-world interaction.
Furthermore, the massive shipment of AI chips in recent years came with a severe shortage of accompanying CPUs, leading to a concentrated catch-up phase benefiting companies like ARM Holdings, Intel, and Advanced Micro Devices.
NVIDIA has also provided a $20 billion revenue guide for its Vera CPU.
However, he warns that a significant portion of this is a catch-up effect.
Once the historical backlog is filled, demand will normalize to incremental levels.
In absolute dollar terms, a NVIDIA Blackwell chip costs around $50,000, while a CPU is around $5,000; even with a higher CPU-to-GPU ratio, the dollar volume for CPUs remains far lower than for AI accelerators.
Memory and AI chips are the main value drivers; CPUs are undergoing a re-rating after being undervalued but will not indefinitely grow faster than AI chips.
Optical interconnects are a long-term bet, but caution is advised on CPO in the near-to-mid term.
While the network and optical interconnect space is another area of high market sentiment, Dylan Patel is cautious on the deployment timeline for co-packaged optics (CPO).
He judges that true mass production of CPO will occur in late 2028 to 2029.
Current manufacturing yields, chip design, and supply chain maturity are not yet at the level required for large-scale deployment.
NVIDIA's Rubin and its successor architecture, Feynman, will still use all-copper solutions, meaning CPO on the GPU side awaits several more chip generations.
He revealed that SemiAnalysis recently advised institutional clients to be more bullish on copper and non-CPO optical solutions in the medium term, while remaining cautious on CPO itself.
This delay extends the windfall period for copper connector companies like Amphenol.
While CPO will happen long-term and copper will eventually be replaced, the timeline has been pushed back, leaving significant near-to-mid-term opportunity for copper.
On-site power generation is set to become mainstream, with diverse innovation paths.
Data center power supply is becoming the most rigid physical constraint on AI growth.
Dylan Patel forecasts new data center power capacity will reach 20 GW this year, 30 GW next year, and 50 GW the following year, representing explosive growth.
He breaks the energy challenge into three dimensions: transmission, generation, and conversion.
Transmission is the hardest to solve due to regulatory and political hurdles.
He predicts that in the coming years, half of the new power for data centers will come from "behind-the-meter" generation—companies building their own power sources rather than relying solely on the public grid.
Current mainstream solutions include combined-cycle gas turbines from suppliers like GE Vernova Inc., Mitsubishi, and Siemens, alongside innovations like reciprocating engines and even repurposed marine or truck engines.
Longer-term, he judges that the levelized cost of solar-plus-storage will fall below gas generation within about two years, with space-based data centers representing an even more distant possibility.
The conversion side is also ripe with investment opportunities across IGBTs, silicon carbide, gallium nitride MOSFETs, solid-state transformers, UPS systems, and supercapacitors.
SemiAnalysis's largest research department now tracks this "Data center, Energy, and Industrial" (DEI) sector, monitoring the deployment of every data center and power plant globally.
Comments