There is still room for growth in AI infrastructure investment, but rising financing costs, semiconductor supply chain constraints, and intensifying competition from in-house chip development are collectively testing the sustainability of this AI boom.
At the 35th annual CASPA conference held in the United States on October 23, Dylan Patel, founder and CEO of SemiAnalysis, Stacy Rasgon, managing director and senior analyst at Bernstein, and Austin Lyons discussed the semiconductor industry outlook, focusing on AI infrastructure investment, supply chain bottlenecks, and the chip competitive landscape.
Patel noted that demand for AI compute remains strong, but capacity constraints in key areas such as substrates and memory are limiting infrastructure expansion. Meanwhile, financing costs have risen sharply, making it increasingly difficult for emerging cloud service providers without long-term customer orders to secure funding. Whether AI investment can continue to advance depends not only on demand but also on the financing capacity and returns of individual projects.
Regarding NVIDIA, Rasgon believes that its product performance improvements can still support higher selling prices, but cloud providers' in-house chip development is changing the competitive landscape. With the advancement of in-house chip initiatives by Google TPU, Amazon Trainium, and companies like OpenAI, NVIDIA may face dual pressure on market share and pricing power even if it continues to grow.
The memory market faces another challenge: HBM demand is strong and industry profit margins are at high levels, but manufacturers' capacity expansion could gradually change the supply-demand relationship after 2028. Over the next two years, financing capacity, supply of key components, and competition from in-house chips will become key variables affecting the pace of AI infrastructure expansion and profit distribution across the supply chain.
Key takeaways:
AI investment still has room for growth, but financing costs are becoming a significant constraint. The U.S. 10-year Treasury yield has breached 5%, and unsecured bond yields for some emerging cloud service providers are approaching 15%. Companies lacking long-term customer orders face greater financing difficulties, and project construction timelines will increasingly depend on financing capacity and investment returns.
Semiconductor supply chain bottlenecks are difficult to resolve in the short term, with substrates and memory being key constraints. Substrates, HBM, optical interconnect components, and cleanroom capacity all face supply limitations. In comparison, power shortages can be partially alleviated through on-site generation, while semiconductor supply chain capacity expansion requires much longer timeframes.
Memory shortages may persist until new capacity is released, with the supply-demand landscape facing a turning point after 2028. HBM demand is robust, and memory industry gross margins are reportedly reaching 88%–90%. However, if new capacity comes online in concentrated fashion after 2028 while demand growth slows, memory prices and profit margins could come under pressure.
NVIDIA still has a performance advantage, but in-house chips are challenging its pricing power. NVIDIA continues to support high prices through performance improvements, but in-house chip initiatives by Google TPU, Amazon Trainium, and companies like OpenAI provide large customers with alternative options. Even if NVIDIA's revenue continues to grow, its market share and profit margins may not increase in tandem.
AI infrastructure costs continue to rise, and investment returns will become the focus of the next phase. The cost of building 1 gigawatt of compute data center capacity has risen from approximately $40 billion in 2024 to over $60 billion. Cloud providers are accelerating in-house chip development to reduce per-unit compute costs, and whether enterprises can continue to expand the model of purchasing tokens through cloud services will affect the long-term returns on infrastructure investment.
How long can AI investment last? Financing costs and supply chain become key constraints
The financing threshold for AI infrastructure construction is rising significantly. The U.S. 10-year Treasury yield has breached 5%, and unsecured bond yields for emerging cloud service providers like CoreWeave are approaching 15%. Volulta issued bonds at a coupon rate of 12% to build clusters for Anthropic, but sold them at 95% of face value, resulting in an actual yield of approximately 15%.
Rasgon noted that hyperscale cloud providers with abundant cash flow in the past, such as Meta and Google, are also consuming significant free cash flow in AI infrastructure construction, and some companies may even need to raise equity financing to support investment in the future.
Patel further pointed out that non-hyperscale cloud providers account for nearly half of the construction demand in the entire market, but such companies typically lack sufficient balance sheet support. Without long-term offtake agreements from hyperscale cloud providers, projects struggle to obtain financing. To this end, NVIDIA has begun supporting certain compute cluster construction through revenue-sharing arrangements, while Broadcom has adopted a similar installment payment approach to supply chips to Anthropic.
This means that financing capacity is becoming an important dividing line for AI infrastructure expansion. Strong demand does not guarantee that all projects will be completed on schedule; for emerging cloud service providers with high capital costs and reliance on external financing, long-term orders and cash flow will directly affect construction speed.
Substrates and memory are harder to rapidly expand than power
Regarding supply chain bottlenecks, Patel listed substrates as one of the most critical constraints currently. Relevant Japanese suppliers are relatively cautious about capacity expansion, limiting the production capacity improvement of accelerator chips, CPUs, and other products.
Memory is another major bottleneck. Rasgon pointed out that producing the same capacity of HBM requires approximately 3–4 times the number of wafers as DDR5, and growing AI chip demand has therefore intensified competition for memory capacity. Meanwhile, components required for optical interconnect, such as lasers and fiber optic connections, also face shortages. Even if CPO accelerates mass production around 2028, related supply may still struggle to meet demand.
The wafer fabrication equipment market is also constrained by cleanroom capacity. Rasgon believes that the forecast of the global wafer fabrication equipment (WFE) market exceeding $200 billion in 2026 is overly optimistic. By improving utilization of existing facilities, the market size could increase from approximately $120 billion expected at the beginning of the year to about $155 billion, but new cleanrooms will not come online until 2027, with more capacity expected to be released in 2028–2029.
In comparison, power shortages have more countermeasures. Patel estimates that nearly 50 gigawatts of data centers are under construction globally in 2027, with approximately 35 gigawatts in the United States. On-site gas turbines, reciprocating engines, and fuel cells can alleviate power constraints to some extent.
Therefore, the bottleneck facing AI construction is not simply a lack of power or land, but the difficulty of synchronizing capacity expansion across multiple key materials and manufacturing stages. In the short term, supply chain constraints and financing costs may jointly affect project delivery speed.
When will the memory shortage end? Supply release after 2028 is key
The current high profits in the memory industry reflect the imbalance between AI demand and limited capacity. Rasgon said that memory industry gross margins have reached 88%–90%, but the market has not given related stocks extremely high valuations as a result, partly because investors worry that concentrated release of new supply after 2028 will cause prices to fall.
What makes HBM special is that its manufacturing consumes more wafer resources. As demand for AI accelerators grows, HBM capacity expansion not only requires increasing its own production capacity but also affects the supply of other DRAM products. In the short term, capacity allocation and expansion speed struggle to keep up with demand; but as new capacity comes online, the supply-demand relationship may also reverse.
When the memory shortage ends depends not on a fixed date, but on when new capacity begins to exceed demand growth. If capacity expansion is concentrated after 2028 while AI demand growth simultaneously slows, current high prices and high profit margins may be difficult to sustain.
Investors need to monitor HBM and DRAM capacity expansion progress, advanced packaging and wafer fabrication capabilities, and changes in cloud providers' capital expenditures. Before supply increases significantly, memory may still constrain AI compute delivery; but over a longer cycle, capacity expansion could also become a factor pushing down prices and eroding profits.
Can NVIDIA's moat hold? Performance advantage faces in-house chip challenges
NVIDIA's core advantage lies in continuously improving chip performance and enhancing customer stickiness through its software ecosystem and development tools. However, as cloud providers increasingly prioritize compute costs, its high prices also provide room for in-house chip development.
Rasgon noted that NVIDIA's valuation relative to the Philadelphia Semiconductor Index is at historically low levels, reflecting market concerns about AI investment sustainability and in-house chip competition.
From a product iteration perspective, NVIDIA is still supporting pricing through performance improvements. From Ampere to Hopper, average selling prices roughly doubled, with performance improving approximately 10 times; from Hopper to Blackwell, prices rose about 50%, with performance improvements reportedly reaching 3–30 times. As long as performance improvements continue to outpace price increases, customers have incentive to pay higher prices for new-generation products.
But competitors are accelerating their pursuit. Patel's research shows that NVIDIA's share of demand for related coprocessor wafers and HBM has begun to decline. Meanwhile, Google TPU is already being sold to external customers, Amazon Trainium continues to improve support for the PyTorch ecosystem, and OpenAI and Anthropic are also advancing in-house chip initiatives.
These initiatives may not immediately replace NVIDIA, but they can provide large customers with alternatives and strengthen their bargaining power. For cloud providers with massive compute demand, the appeal of in-house chips lies not only in reducing procurement costs but also in reducing dependence on a single supplier.
Therefore, the key test for NVIDIA is not whether it can continue to grow, but whether it can continuously justify its price premium through performance advantages. Even if the AI market expands overall, NVIDIA's revenue, market share, and profit margins may not grow in tandem.
Enterprise AI adoption: purchasing tokens is more realistic than building compute
On the enterprise application side, the three analysts believe that most enterprises will not purchase large amounts of hardware to run private models themselves, but will instead purchase tokens through cloud services—that is, calling cloud-based AI inference capabilities and combining them with their own data and business processes. This is similar to the past trend of enterprise IT shifting from self-built infrastructure to cloud computing.
Currently, AI-assisted programming is one of the more mature enterprise-level applications. Rasgon shared that with the help of Claude, a financial model that would normally take a week to complete was built in approximately 1 hour and 20 minutes, involving 10 years of historical data, multiple business restructurings, and logical inference from management earnings call content.
The value of such applications is first reflected in saving time and reducing repetitive work, rather than necessarily immediately showing up in revenue. Their return on investment still needs to be validated through actual business results, but enterprises calling models through the cloud also means that demand for AI infrastructure will increasingly depend on the usage of inference services.
If enterprise applications continue to expand, cloud providers will have incentive to continue investing in compute; but if construction costs and financing costs rise faster than revenue, the returns on infrastructure investment will still come under pressure.
Full transcript below:
Moderator Austin Lyons:
Let's start from a more macro perspective, first discussing demand, then supply. On the demand side, AI revenue is growing very rapidly, but the growth rate has begun to slow. Currently, the combined revenue of the entire AI industry is still only a small fraction of total industry spending, so the market is scrutinizing every revenue data point extremely carefully.
On the supply side, compute capacity for the next several years has already been booked out, and prices remain firm. From a semiconductor analyst's perspective, how should we understand these two signals? And where will value in the supply chain ultimately flow?
Today, we have two important figures from the semiconductor industry: Dylan Patel is the founder and CEO of SemiAnalysis, one of the most influential research institutions in the AI infrastructure field; Stacy Rasgon is a managing director and senior analyst at Bernstein who has been analyzing semiconductors on Wall Street for over a decade. I am today's moderator Austin Lyons, currently Chief Strat chief analyst and formerly a principal engineer at Intel.
Today, we will discuss the future of the semiconductor industry, looking ahead to 2027 and 2028, covering financing, GPUs, CPUs, and wafer foundry, among other areas.
1. AI infrastructure investment: will financing costs slow expansion?
Moderator Austin Lyons:
Recently, the cost of capital has become a focus of market discussion. Hyperscale cloud providers are investing enormous amounts of money, and free cash flow has trended negative. At the same time, the U.S. 10-year Treasury yield has breached 5%, and bond market yields for companies like CoreWeave are approaching 15%.
Should we be concerned about this? As companies continue to increase investment and financing costs keep rising, how much impact will this have on AI infrastructure construction? Let's start with Stacy.
Stacy Rasgon:
There are indeed many limiting factors currently slowing the pace of AI infrastructure construction, including power, supply chain, semiconductor manufacturing equipment, and now possibly financing as well.
From a stock performance perspective, although related companies' share prices are still up considerably from earlier levels, they are clearly below their highs. The market is sending a signal: investors are worried about great uncertainty regarding the speed and sustainability of infrastructure construction, and also worried that industry growth may be approaching a peak. Financing is clearly one important issue, perhaps even one of the most important.
You just mentioned that hyperscale cloud providers were once among the most cash-generative enterprises in human history, but now they are consuming large amounts of cash and may need to issue equity in the future. Meta recently adjusted its equity structure, partly to provide room for such financing arrangements. Google and other companies have also conducted financing.
Taking CoreWeave as an example, its bond yield is approaching 15%. It's important to note that this does not represent the actual financing cost of all the company's current debt, but rather the trading yield of its unsecured bonds in the market. But if the company re-entered the market today to issue unsecured bonds, the financing cost would likely be around this level. That is a fairly high cost.
However, I think that for at least the next year or two, the industry can largely continue to operate. More importantly, related revenue must continue to grow to gradually alleviate market concerns about financing.
Taking the compute leasing market as an example, supply is currently extremely tight, and the general refrain we hear is that no one can get enough compute. Some companies lease compute through short-term contracts, and by some estimates, revenue per gigawatt of compute could even reach around $50 billion.
Therefore, it is not difficult to imagine a scenario where, even with high construction costs, related revenue is still sufficient to cover costs or even exceed them. But the question is: how long can this revenue level be sustained? Ultimately, it depends on how strong demand is relative to supply and how long the supply-demand gap can be maintained.
These questions may require Dylan to provide deeper judgment from an industry supply-demand perspective. But financing is undoubtedly one of the many current constraints.
One more thing worth noting: constraints slowing growth may not be entirely a bad thing. They may prevent the industry from over-expanding and avoid runaway investment.
A similar situation occurred during the internet bubble. At that time, large amounts of dark fiber were laid—fiber that had been installed but not yet put into use—and much infrastructure was ultimately overbuilt, with some still not fully utilized today.
Today's construction constraints may actually allow supply and demand to match more reasonably and enable industry growth to last longer. If we expanded wildly without any constraints today, we might face a more severe crash risk in the future.
So, I don't think all constraints are bad. Of course, how the market views these constraints is another matter, and they have clearly affected the short-term performance of related stocks.
Dylan Patel:
I agree with Stacy's view. Looking back at infrastructure construction over the past few decades, every time the industry expanded, companies tended to double- or even triple-order, ultimately building too much capacity. The supply chain has never been good at forecasting demand.
Now, financing is playing a self-regulating role to some extent. A year or two ago, as long as a cloud service provider was willing to pay a high enough interest rate, it could usually obtain funding; but now, without sufficiently high credit quality long-term offtake agreements, some emerging cloud service providers cannot build compute clusters.
Hyperscale cloud providers can certainly push free cash flow to near zero, can continue to borrow, and since their profits are still growing, their construction capacity will increase further. But they cannot invest indefinitely. Next year, they cannot raise capital expenditures to over $2 trillion, or even $2.5 trillion. Funding capacity will ultimately limit spending scale.
Now look at emerging cloud service providers (Neoclouds). Non-hyperscale cloud providers account for nearly half of the construction demand in the entire market. If they don't have offtake commitments from hyperscale cloud providers, how can they obtain financing?
Today, even with offtake commitments from customers like Anthropic or OpenAI, the market may require related debt to offer yields of 13%–15%. Volulta is an example: the company is building a compute cluster for Anthropic, with a bond coupon rate of approximately 12%, but issued at only 95% of face value, resulting in an actual financing yield approaching 15%.
If a company does not have offtake commitments from hyperscale cloud providers or large AI labs, how can it build compute to serve small and medium-sized enterprises and startups? Frankly, without the support of these large customers, financing would be extremely difficult.
This is also one of the strategies Elon Musk has adopted: build large amounts of compute first, then lease it to startups and AI labs at higher prices through short-term contracts. But overall, financing constraints will indeed suppress industry expansion.
NVIDIA has realized this is a major issue. There are a few large customers in the market who can buy as much compute as their own funds allow; but there is another half of market demand—how to serve these customers?
NVIDIA has therefore begun adopting some innovative financing structures. For example, in projects like Sharon AI and Fermis, NVIDIA supports project financing by sharing compute cluster revenue. The arrangement is roughly as follows: NVIDIA leases the cluster at a price that guarantees the cloud service provider essentially breaks even, and if the cloud service provider leases compute to other customers at higher prices, NVIDIA shares a portion of the revenue.
Similar arrangements are increasing. When Lambda builds a compute cluster for Anthropic, it also needs to sign a revenue-sharing agreement with NVIDIA, with a sharing ratio of approximately 10%.
This shows that whether emerging cloud service providers can obtain sufficient capital to complete construction and ultimately even expand to the point of oversupply is constrained by financing conditions.
We also see that industry financing methods are shifting from long-term loans and long-term debt to more flexible arrangements. When Broadcom supplies chips to Anthropic, it adopted a lease-to-purchase-like approach, extending payment terms over several years.
For companies that cannot rely on their own balance sheets like hyperscale cloud providers, obtaining compute requires NVIDIA, Broadcom, or AMD to provide some financing support or credit guarantees for clusters.
Therefore, financing is an important factor limiting AI infrastructure expansion. Hyperscale cloud providers themselves also have enormous on-balance-sheet and off-balance-sheet commitments. For example, Meta's commitments in computing and data centers are reportedly reaching $1.4 trillion.
These projects can be funded in various ways: companies can borrow to buy data centers, or lease data centers or related equipment from other companies, and the party providing the infrastructure can then finance based on that. But getting the entire industry to continue expanding significantly on this basis is becoming increasingly difficult.
The capital market is even beginning to feel uneasy about Meta's financial arrangements. Meta's own financial condition does not mean it cannot afford these investments, but the sheer scale of such commitments is enough to make the market more vigilant.
Ultimately, AI infrastructure construction has three interrelated core conditions: land, power, and shell facilities; chips; and financing. Constraints in any one link can slow overall expansion.
2. Where are the semiconductor supply chain bottlenecks in 2027–2028?
Moderator Austin Lyons:
The financing situation is relatively clear: hyperscale cloud providers are relatively capable of bearing investment, but emerging cloud service providers face greater pressure, and the industry is trying new financing structures.
If we temporarily set financing aside and return to traditional supply chain bottlenecks, which links will actually constrain industry growth over the next two years?
ASML says its 2028 capacity is already sold out; TSMC's demand also consistently exceeds supply, even as it expands 2-nanometer process capacity. The market also widely believes power is an important bottleneck.
Dylan, you recently mentioned that nearly 50 gigawatts of data centers may come online next year. Could you rank the various links in the semiconductor supply chain by degree of constraint?
Dylan Patel:
Within the semiconductor supply chain, apart from memory, I think the biggest bottleneck is substrates, including packaging substrates, PCB materials, and other related materials.
These links have suffered from underinvestment for a long time. Many related material suppliers in Japan operate conservatively and are unwilling to expand capacity on a large scale, so over the next two years, whether AI accelerators, CPUs, or other components in the supply chain, all will be significantly constrained.
In my view, substrates are one of the most severe bottlenecks currently, with constraints even exceeding land, power, data center shell facilities, and some semiconductor manufacturing equipment.
The second-largest bottleneck is memory.
The third category of bottleneck is optical interconnect, including co-packaged optics (CPO) and near-packaged optics (NPO). It is not easy for the related industry to significantly increase output. Even if all companies want to adopt CPO by 2028, or one large company wants to fully deploy it by then, they cannot obtain enough lasers, fiber optic connection components, and other supporting parts.
As for land, power, and data center shell facilities, power is indeed an important constraint. But our research found that the market may be overestimating the constraints facing data center construction next year.
It is estimated that nearly 50 gigawatts of data centers are under construction globally next year, with approximately 35 gigawatts in the United States, and the rest distributed in Australia, India, Malaysia, and other regions. In fact, data centers are being built rapidly around the world.
Although power remains tight, companies can take various measures to alleviate constraints, such as deploying gas turbines, reciprocating engines, or fuel cells on-site at data centers.
Therefore, although power and data center construction conditions do have constraints, their impact may be lower than substrates, memory, lasers, and other optical supply chain components.
Power supply can be increased through relatively direct methods in a shorter time frame, while the semiconductor supply chain involves much more complex manufacturing processes and material systems, and capacity expansion requires longer timeframes.
Stacy Rasgon:
This judgment is very interesting. Both NVIDIA and Broadcom have given quite high growth guidance. Jensen Huang has said NVIDIA's revenue could grow at least 70%; Broadcom's growth expectation is even 100%. But these guidances themselves are already constrained, and the constraints companies mainly consider seem to be land, power, and data center construction conditions.
Perhaps this is because they are semiconductor companies themselves, are relatively clear about how many chips can be produced, and therefore focus more attention on data center construction.
I agree there are some solutions for power and land. However, even on-site power generation may not be achievable quickly. How long does it take to order a gas turbine today? Musk even said he is prepared to manufacture turbine blades himself to bypass supply constraints.
The memory market is particularly worth watching. Memory stocks currently have very low valuations, with many trading at forward P/E ratios of only two, three, or four times, while memory prices have already risen significantly.
I don't directly cover memory stocks, but take SanDisk as an example: its quarterly earnings per share guidance is even higher than the company's stock price from about two years ago. This shows the current price increase is very strong.
However, memory stocks generally trade at single-digit P/E ratios because the market typically prices stocks this way when the industry cycle is near its peak. Investors worry that, especially after entering 2028, new supply will begin to be released and memory prices will fall sharply.
I keep thinking: what if this doesn't happen?
Look inside an AI rack: about 70%–80% of the silicon area is actually memory, mainly DRAM. High Bandwidth Memory (HBM) has another special issue: producing the same capacity of HBM requires approximately three to four times the number of wafers as ordinary DDR5 DRAM.
This means HBM production consumes a large amount of wafer resources, squeezing capacity that could otherwise be used to produce ordinary memory needed for PCs, smartphones, and other products.
3. Memory and manufacturing equipment: shortages may persist until 2028
Stacy Rasgon:
HBM demand is very large, which leads to another bottleneck: semiconductor manufacturing equipment, that is, the equipment used to produce these chips.
This week, one company released a preliminary outlook for next year's wafer fabrication equipment (WFE) market, forecasting a market size exceeding $200 billion. I found this forecast quite surprising because I think it is difficult for the market size to reach a higher level, and even whether this figure can be achieved is questionable. The reason is simple: we don't have enough fabs.
Cleanrooms are almost entirely occupied. Companies have been maximizing utilization of existing facilities, which is why at the beginning of the year the market estimated WFE size at about $120 billion, but this year it may ultimately reach $155 billion. The growth mainly comes from more full utilization of existing facilities rather than large-scale new construction.
But there is ultimately a limit to the utilization of existing facilities. Companies must build new cleanrooms, which takes years. New cleanrooms will begin to come online in small amounts in 2027 and increase further in 2028 and 2029. Only then can we more clearly judge whether supply and demand will become imbalanced.
If demand continues to grow at the pace currently expected by the market, memory may remain in a state of prolonged supply tightness.
Substrates, PCBs, and other materials face similar problems. Equipment suppliers have said that short-term supply constraints have already affected their ability to increase output. What is constrained is not only PCBs and semiconductor chips, but also seemingly ordinary components such as valves and mass flow controllers. Shortages of these basic components also affect other links in the supply chain.
During the pandemic, semiconductor equipment companies and most companies in the supply chain learned some lessons, and now manage supply issues more maturely than before. But many links are still constrained, and these problems must be resolved first before capacity constraints in other links can be alleviated.
Therefore, the supply chain actually has layer upon layer of constraints. I think the shortage situation will not disappear in the short term. This year is not a big problem, but next year may still be tight, and 2028 will likely not be loose either. As for 2029, it depends on demand at that time. Overall, I expect supply tightness to persist for some time.
4. Pricing power in the semiconductor industry is changing
Moderator Austin Lyons:
If supply remains tight over the next few years and memory industry profit margins have already reached over 80%, even 88%–90%, will companies across the entire supply chain continue to raise prices and capture higher profits?
Stacy Rasgon:
The semiconductor industry has a very interesting characteristic: in the past, people almost took continuously falling prices for granted. In fact, the semiconductor industry historically was an industry that severely destroyed economic value, unable to cover its cost of capital for a long time, with most of the value created flowing to consumers.
Many years ago, when I worked at McKinsey, I participated in a study on value creation in the semiconductor industry. We used the classic definition of economic profit, comparing companies' return on invested capital with their weighted average cost of capital, and aggregated across the entire industry. The study covered about 150 companies, 12 sub-sectors, and one complete industry cycle.
The initial time range of the study was approximately 1996 to 2004. The results showed that the entire industry cumulatively destroyed about $5 billion in economic profit. If Intel were excluded, the destruction would reach about $50 billion. At the time, that was a fairly astonishing figure.
In other words, the value created by the industry was not enough to cover the cost of capital, and economic value actually flowed to consumers. That was an era when buying semiconductor products was very cost-effective.
After 2008, the industry underwent consolidation and concentration, and companies began to gradually improve profitability, at least slowly becoming able to cover the cost of capital.
The pandemic changed the industry's understanding of its own pricing power. Companies discovered that they actually had strong pricing power. Previously, the industry had not truly realized this.
During the pandemic, input costs such as raw materials continued to rise, but none of the many companies I tracked saw gross margins decline because of rising costs. They all raised prices, and the magnitude of price increases was enough to offset cost increases.
For example, if a company originally had a gross margin of 60%, and costs rose by 100 units, it could raise prices by 160 units to maintain the original gross margin. Customers also accepted these price increases.
I think pandemic-era inflation actually helped the semiconductor industry improve profitability. In 2023 and 2024, industry shipments were roughly flat compared with 2019, but overall average selling prices (ASP) were about 50% higher, even excluding memory.
Today, this pricing power is even more evident. Total semiconductor industry revenue this year is expected to reach about $1.5 trillion, possibly even higher, with year-over-year growth approaching 100%. Of that, about 70%–80% of the revenue increase comes from higher memory average selling prices, and a considerable portion of the rest also comes from price increases in non-memory products.
In comparison, shipment growth is much smaller. The industry is exercising its pricing power more fully, and customers have generally accepted these prices because, in a tight supply situation, they often have few alternatives.
Therefore, as long as supply remains tight, I think this situation can continue.
Dylan Patel:
In the semiconductor industry, different companies use pricing power in different ways. Some companies raise prices sharply during booms but suffer severe blows during downturns; others are able to maintain relatively stable profit margins throughout the cycle.
Take TSMC as an example. Over the past decade, its gross margin has steadily increased from about 45% to about 60%. It can indeed raise prices during booms, but during downturns it has not significantly given back previously gained profits.
The memory industry is different. Just about two years ago, the industry was in one of its worst downturns since the internet bubble, with gross margins even negative. Now, memory manufacturers' profitability has improved dramatically.
Therefore, pricing power at different links in the supply chain depends on their respective market structures. TSMC has a near-monopoly position and maintains long-term relationships with customers, so even if it has the ability to raise prices, it has not raised them as sharply as some other companies.
Memory manufacturers are different. If customers like Apple exert great pressure on suppliers during downturns, then when the cycle reverses and supply falls short of demand, suppliers will also raise prices. What we are seeing now is exactly this change in cyclical bargaining power.
5. Compute construction costs are soaring—who bears the high costs?
Dylan Patel:
In 2024, the total cost of building 1 gigawatt of compute was approximately $40 billion, covering chips, land, power, and data center facilities. Today, this cost has exceeded $60 billion.
Of course, not many companies actually built 1 gigawatt of compute in 2024, while by 2026 more and more companies will begin building compute facilities at such scale. At the same time, the computing power each gigawatt of compute can provide is also increasing.
But the question is: such a gigawatt-scale compute cluster may be extremely profitable.
I mentioned earlier that cloud service providers often must obtain long-term offtake commitments from hyperscale cloud providers to finance clusters. This means they need to lock in prices in advance, which also limits their room to earn excess profits.
Musk took another approach: if chips, land, power, and data center facilities are all constrained, then he can build the infrastructure first and sign lease contracts only when it is actually put into use. In this way, when compute supply is very tight, he has the opportunity to lease at higher prices rather than locking in lower prices in advance.
At the same time, AI labs themselves can also earn enormous revenue from inference services. According to relevant disclosures, Anthropic's revenue per gigawatt of inference compute may reach $70–$80 billion.
But these revenues do not mean profit margins can remain high indefinitely. If a company obtains compute at a cost of $15 billion per gigawatt, profit margins may reach 75%–80%; if the cost rises to $50 billion, profit margins will fall significantly.
Therefore, who ultimately bears these costs remains a question.
NVIDIA must also face this pressure. Although its profit margins have remained broadly stable or even slightly declined, it has had to absorb some costs from memory price increases as well as substrates, optical components, and other supply chain links.
The result is that NVIDIA has raised prices to customers significantly, but its own profit margins have not risen by the same proportion.
This makes profit distribution across the supply chain particularly important: who can protect margins, who can further expand profits, and who has to bear the pressure of rising costs? These questions are changing the industry's pricing landscape in real time.
In the future, customers may sign similar contracts at different prices. Companies with strong sales capabilities and the ability to seize opportunities when supply and demand are tight may obtain very high prices.
Moderator Austin Lyons:
This is very similar to the pandemic-era change Stacy just mentioned: the semiconductor industry gradually realized it could charge for the value it creates.
Dylan, you also mentioned that system suppliers like NVIDIA must decide, when prices for various components are rising, whether to pass costs on to customers or bear the costs themselves and maintain profit margins.
The question is: infrastructure construction will clearly continue over the next few years, with limited supply and strong demand, but financing is becoming increasingly difficult. So will large cloud providers and model companies like Meta one day decide that compute has become too expensive to bear and begin resisting further price increases?
Stacy Rasgon:
When evaluating these investments, you cannot look only at costs and profit margins, but also at what companies get after spending the money.
Take NVIDIA as an example: each generation of products has a higher selling price, but the performance improvement customers receive is often even greater. This is very important.
From Ampere to Hopper, GPU average selling prices roughly doubled, but performance improved about 10 times. From Hopper to Blackwell, the situation is harder to compare directly because the two generations differ significantly, but one can roughly estimate that average selling price per GPU rose about 50%, while performance improved 3–30 times, with actual improvement possibly even greater considering subsequent software optimization.
From Blackwell to Rubin, rack costs may rise sharply again. According to reference data provided by NVIDIA, Blackwell rack costs are about $25 billion and Rubin about $40 billion, but performance improves about 10 times.
Therefore, the key is not whether the construction cost per gigawatt of compute rises from $25 billion to $40 billion, or from $40 billion to $60 billion, but how many tokens and how much computing power customers can obtain for every dollar they spend.
As long as performance per unit cost continues to improve, such investment remains attractive.
But if NVIDIA launches a new generation product in the future with performance improvement of only 20%, I would become very worried about its pricing power and profit margins.
NVIDIA's gross margin exceeds 70%, and it is built on a massive revenue scale. The market has been asking: in the face of competitors and other GPU suppliers, how can it continue to maintain such high profit margins?
My view is that as long as NVIDIA can continue to make performance improvements significantly exceed average selling price increases, there is no need to worry too much about profit margins. Because as product complexity rises and costs increase, selling prices must also rise; the key is the relationship between performance and cost, which must always remain favorable to customers.
If the pace of innovation slows, I would become very worried.
I remember attending the Hot Chips conference around 2023. At that time, NVIDIA chief scientist Bill Dally gave a keynote speech, and one point left a deep impression on me: over the past decade, GPU performance increased about 1,000 times. NVIDIA achieved this progress through a variety of different technical means, and Dally said the goal for the next decade is to increase performance by another 1 million times.
If NVIDIA can continue along this trajectory, I would be less worried about its profit margins. But if it cannot maintain the pace of innovation, or if other competitors quickly catch up, the situation would be completely different.
Ultimately, someone has to bear these costs. Ideally, customers are willing to pay for the value they receive. But if costs rise at an unsustainable pace, suppliers will have to start bearing some of them, and the profit struggle among different links in the supply chain will become more intense.
6. Rise of in-house chips: how long can NVIDIA's competitive advantage last?
Moderator Austin Lyons:
Dylan, more and more companies are designing their own chips. The OpenAI case is particularly noteworthy. Over the next two years, how should we view GPU development and the rapid growth of various XPUs and AI accelerators? Many startups are trying different chip architectures—how will this change the competitive landscape?
Dylan Patel:
More and more companies are trying to design their own chips; this is already a clear trend. OpenAI demonstrated an important fact: even a company designing a chip for the first time can produce a product with excellent performance.
OpenAI once invited us into its lab. We used SemiAnalysis's open-source inference benchmark InferenceX to test a variety of open-source models, and the results showed that its first-generation chip outperformed NVIDIA's Blackwell.
This chip made extensive use of AI tools in its design and programming process. Anthropic has also begun advancing its own chip project and hopes to complete development in half the time OpenAI took. These goals are quite aggressive, but if models can significantly accelerate the design process, they are not entirely impossible.
At the same time, Google has already begun selling TPUs externally, Amazon's Trainium has also become quite mature and supports PyTorch well. Meta and Microsoft also have very aggressive chip roadmaps.
There is a strong economic incentive behind this. As we mentioned earlier, the cost of building 1 gigawatt of compute has risen from $40 billion to over $60 billion. If the profits of chip design companies such as NVIDIA, Broadcom, and Marvell could be eliminated, the cost of the same scale of compute might be only around $20 billion.
In other words, if the profit cost in the chip design link can be significantly reduced, then in theory the money saved per gigawatt of compute could support building more compute. For companies that need to run large inference workloads, this economic benefit is very obvious.
Therefore, NVIDIA must maintain extremely fast innovation. It cannot merely double performance per dollar each year, otherwise other competitors may quickly catch up.
If NVIDIA can continue to maintain this pace of performance improvement, it certainly has reason to continue earning high profits. But if large customers can design their own chips, deploy them, and expand production scale, while obtaining memory, TSMC wafers, substrates, and optical components at prices close to supply chain cost, then they have substantial room to cut prices.
If the supply chain further disintegrates and NVIDIA or Broadcom no longer hold the same dominant position as before, the situation could change significantly. This is also one reason the market is worried about both companies' valuation multiples being at historical lows.
For example, will Meta reduce its dependence on Broadcom? Will Google do the same? Will these large customers eventually stop purchasing NVIDIA chips and shift entirely to internal designs?
Currently, component suppliers themselves may also gain strong pricing power. The real question is whether NVIDIA and Broadcom can continue to maintain their currently astonishing profit margins. At least for the next two years, I think they still have an opportunity to do so, but beyond two years it becomes very difficult to say.
Chip design itself remains very complex, but AI models are significantly reducing the difficulty of software development and accelerator programming.
Take our InferenceX benchmark as an example: getting NVIDIA's inference system running properly took about six months; AMD took six to nine months; Google and Amazon both took only a few months; and OpenAI's chip took only about a month and a half.
From a chip being able to work to software being able to run on the chip, this process is rapidly shortening. Now, companies are beginning to shift their attention to chip design itself.
Will chip design gradually become a more widespread and standardized capability, like software development? If the answer is yes, the number of companies capable of designing chips could increase substantially.
Of course, chip design still requires fixed costs such as masks and electronic design automation (EDA) tools, which alone can amount to $50 million. But if AI continues to lower the barriers to design and programming, the number of companies participating in chip design may still increase.
Once chip design becomes more widespread, industry profit margins could decline. We have already seen some software-as-a-service (SaaS) companies undergo market revaluation; at the same time, NVIDIA and Broadcom's earnings are still growing at more than double annually, but their stock performance has been relatively stagnant.
This is precisely one of the most noteworthy questions for the semiconductor industry's future.
7. Can XPU startups challenge GPUs? NVIDIA's market share is changing
Moderator Austin Lyons:
So how will the GPU and XPU markets develop over the next two years? More and more AI accelerator startups are trying different architectures—will the competitive landscape change significantly?
Dylan Patel:
For XPU companies, I think there are two different paths.
If you are a hyperscale cloud provider, you can take a relatively steady design route, produce something similar to Google or NVIDIA products, and rely on your own scale to drive adoption.
But if you are a startup, you must try a very aggressive technical route that is clearly different from existing solutions. Looking back at the first generation of AI hardware startups, one important reason Cerebras and Groq were relatively successful is that they took bolder technical paths.
For the startups emerging now, I think only those with technical solutions unique enough that they cannot be easily replicated by NVIDIA, Google, or AMD truly have a chance to stand out.
The good news is that a large amount of capital is flowing into this field, and various aggressive architectures and design ideas are being tested.
At the same time, NVIDIA's share in the supply chain has already begun to change. From 2023 to 2024, NVIDIA occupied a very high proportion of HBM and advanced packaging capacity; by 2025, this growth began to slow, and in 2026 its share declined further. According to our judgment, this trend may continue next year.
But this does not mean NVIDIA's profits will immediately decline. The overall market is growing too fast, and even if share falls, NVIDIA's earnings next year may still double.
Google, Amazon, and AMD are all gaining share. Here, you cannot look only at chip count, because different chips vary greatly in size and value. More meaningful indicators include advanced packaging wafers, HBM usage, and spending on wafer foundry and memory.
All these indicators show that competitors are gradually expanding their market share.
Looking ahead to 2028 and 2029, NVIDIA's growth rate may slow, and it may not be able to maintain annual revenue growth of 100%. At that point, the market will need to judge: will slower growth further lead to share decline, or can NVIDIA continue to lead through a new-generation architecture after Rubin?
NVIDIA must continue to accelerate innovation. Its one-generation-per-year chip update cadence is not only for its own development but also to make it difficult for competitors to catch up.
NVIDIA's moat is not just hardware. CUDA software, a massive developer ecosystem, already deployed infrastructure, and its own balance sheet and financing capacity all constitute competitive advantages.
NVIDIA is also using AI to accelerate its own chip design. It is not unaware of competitive pressure, nor is it failing to adopt new technologies. Ultimately, this is a race.
Jensen Huang is clearly highly vigilant about competition. He has learned some lessons from Intel's history and always pays attention to competitors' movements.
Stacy Rasgon:
This also explains why NVIDIA's valuation is at such a low level.
About nine to ten months ago, I wrote a research report comparing NVIDIA's relative valuation with the entire semiconductor industry. At that time, over the past decade, NVIDIA's relative valuation had been lower on only 13 trading days.
Now, its relative valuation is even lower than it was then.
Investors are worried not only about whether AI demand can continue. If all AI demand disappeared, then both NVIDIA and other companies would be severely affected.
But within the semiconductor industry, NVIDIA's biggest threat clearly comes from competition. It is currently far ahead of peers, but all competitors want to seize its market.
This is also a consistent characteristic of the semiconductor industry: as long as one company succeeds, there will always be others trying to challenge it. NVIDIA must ensure it always stays ahead, and as the industry develops, this will only become harder.
8. Enterprise AI applications: will more companies buy chips themselves in the future, or directly purchase tokens?
Moderator Austin Lyons:
We discussed GPUs, XPUs, and competitors continuously expanding market share. Our discussion focused mainly on data centers and hyperscale cloud providers.
So for companies like NVIDIA, is enterprise on-premises compute also an important growth opportunity?
Stacy Rasgon:
NVIDIA now distinguishes revenue from hyperscale cloud providers and non-hyperscale cloud providers, but this division is not entirely accurate. Some AI labs may be classified as non-hyperscale cloud providers, and hyperscale cloud providers' spending may also indirectly support these labs, so there is overlap between the two.
Roughly speaking, the two types of customers currently account for about half each. Revenue contributed directly or indirectly by AI labs may account for about 20% of NVIDIA's total revenue. That is still a very large proportion.
Therefore, NVIDIA's revenue sources may be more diversified than we previously imagined.
But ultimately, enterprise adoption of AI is key to whether this industry can continue to develop. If enterprises do not truly use AI, the entire industry's long-term growth lacks support.
It is still early stage, but I think at least one core application with clear value has emerged on the enterprise side: programming.
Revenue from AI programming agents is growing rapidly. Today, almost all software developers use AI-assisted programming, and few still write code entirely by hand.
The value of enterprise applications may not necessarily be directly reflected in revenue growth, but may also be reflected in productivity improvements, which are harder to measure.
I myself am a financial analyst and need to build financial models. A few months ago, our team tried using Claude to do this kind of work. The initial results were not ideal, but later a colleague thought the tool had improved significantly.
So we arranged a team meeting and had it build a financial model from scratch for a company we already cover.
The result was surprising: it took only one hour and twenty minutes to complete a model. The model covered ten years of historical data, handled multiple business restructurings and segment changes, and attempted to use logical inference for forecasts based on the CEO's remarks on earnings calls, rather than simply generating random numbers.
This work, if assigned to one analyst on the team, might take a week; if I did it myself, it might take three months. But Claude took only one hour and twenty minutes.
This means the same team can complete more work, or even complete more tasks with fewer people.
Of course, this also raises other issues. If AI becomes widely adopted in enterprises, could it cause a sharp rise in unemployment? I don't know the answer, but it is indeed worth considering.
Based on my actual experience so far, I increasingly believe AI applications can bring real returns. However, this is still an early market. Many enterprises are still in the experimental stage, unclear about how to use AI, and lack sufficient expertise to judge specific needs, so adoption will take time.
Overall, I am positive about what I have seen so far.
Dylan Patel:
I would like to add one point: regarding enterprise AI applications, I do not think most enterprises will purchase large amounts of hardware and run their own models internally.
Model inference has obvious economies of scale. For most enterprises, the more reasonable approach is to directly purchase tokens—that is, call cloud-based model inference services and combine those capabilities with their own data.
I think this will be the main way most enterprises consume AI in the future.
This is very similar to enterprise IT development over the past two decades. Twenty years ago, enterprises typically bought infrastructure and deployed systems themselves; today, most IT spending has shifted to cloud computing. In the future, most enterprise spending on AI may also go toward purchasing tokens, rather than buying compute and running open-source models themselves.
Even pharmaceutical companies and hedge funds may mainly pay for direct model use rather than buying hardware to run open-source models themselves. Of course, there will still be some suppliers in the market that specifically procure compute, deploy models, and provide services to customers.
There is much more worth exploring in this discussion, but we are already overtime. Today's exchange was very interesting, and thank you all for participating.
Comments