About this article
This article reconstructs the content of "Enterprise AI Investment Trends and Shifting Infrastructure Demand — A Hyperscaler Perspective," presented at an invited expert seminar of the Bank of Korea Research Department in September 2026. All data is based on public sources (earnings releases, IR disclosures, industry research, as of August 2026). Views are the author's own and do not represent any affiliated organization.
Summary
The center of gravity in AI demand is shifting from building models to using them — and that shift is changing the very nature of semiconductor demand. Statistics and disclosures lag by quarters; enterprise contract and architecture decisions happen in the field six to twelve months earlier. This article organizes those leading signals around four questions: ① Is this investment a bubble or a structural shift? ② How are enterprises actually using AI? ③ What does the training-to-inference-to-agents transition mean for semiconductor demand? ④ Where is the next axis of demand?
1. Bubble or Structural Shift — Four Tests
Start with scale. Big 4 hyperscaler capex for 2026 is $650–725B, up roughly 77% year over year — around 2.2–2.4% of US GDP. Against historical infrastructure buildouts, that already exceeds the dot-com telecom ratio (~1.2%) but falls short of the railroad peak (~6%). Not unprecedented in scale, but unprecedented in speed. And there is one decisive difference: railroads and power grids depreciate over decades, while AI servers depreciate over five to six years. The verification cycle is short — the sustainability of this cycle depends on how quickly revenue recovery is proven.
Hence four tests.
Backlog — contracts first, construction later
AWS's remaining performance obligations (RPO) stand at $496B, up $132B in a single quarter. RPO is signed-but-unrecognized revenue — a leading indicator for capex and semiconductor exports. Unlike the dot-com era, signed contracts now exceed supply capacity.
The circularity critique — real, but not yet cycle-invalidating
Chip suppliers invest in model companies, which buy compute with that money — the circular structure is real. But three things differ from dot-com vendor financing: paying end demand (subscriptions, API revenue) actually exists and is growing fast; equity investments are separated from revenue recognition; and most of the backlog growth comes from ordinary enterprise customers. Circularity deserves monitoring, but it is not yet grounds to dismiss the cycle.
Revenue recovery — is the investment coming back?
AWS maintains a 39.4% margin while disclosing $25B+ annualized AI revenue growing at triple digits. "AI and core cloud are pulling each other" is the supplier-side explanation — and so far the numbers support it.
End demand — adoption passed the threshold, proof is halfway
74% of large enterprises have reached production deployment, escaping "pilot purgatory." Yet about half cannot measure ROI consistently, and 48% bolted AI on without redesigning workflows. The bottleneck is organizational, not technical.
The most honest demand indicator: how customer questions changed
| Year | Typical question in enterprise meetings | Demand stage |
|---|---|---|
| 2023 | "Can we build something like ChatGPT too?" | Curiosity & exploration |
| 2024 | "Which model is best? What are the benchmark scores?" | Model-centric thinking — PoC proliferation |
| 2025 | "The PoC worked — now what about security review and cost control in production?" | Transition to operations |
| 2026 | "How much authority can we give an agent? Why is our token bill exploding?" | Operating at scale |
Generalized pattern from the author's field experience.
Model questions have nearly disappeared; only operations questions remain. Models themselves are becoming commodities, and differentiation has moved to data, workflows, and governance. The maturity of the questions is the maturity of the spending — exploratory one-off PoC budgets and operating expenses differ fundamentally in persistence, and that is the micro-level background of the backlog growth.
2. How Enterprises Actually Use AI — The Token Consumption Explosion
Enterprise infrastructure choices start with APIs and then differentiate by workload as usage grows — not a wholesale retreat but a portfolio: regulated data to private endpoints, predictable high-volume workloads to multi-year commitments, real-time UX to user-proximate deployment. The moment a company converts to a multi-year commitment is the moment its AI spending is reclassified from experiment to operating expense — the micro-level substance of that $496B backlog.
Meanwhile, consumption is exploding structurally. Unit prices are flat or falling, yet per-query consumption keeps rising, for four structural reasons: ① long context — feeding entire documents and codebases has become standard; ② retrieval (RAG) — every query carries dozens of pages of grounding documents; ③ agent loops — plan, call tools, review, retry: one task becomes dozens of model calls; ④ reasoning chains — models spend tens of thousands of tokens "thinking" before answering. One large health insurer's monthly token consumption grew from 3 million to 150 million in a year — 50x. 93% of enterprises have exceeded their AI budgets (McKinsey), and the cause is volume, not unit price.
Jevons paradox — unit price and consumption moving in opposite directions
Cut a customer's unit price by 30% and months later the total bill is higher — the saved budget gets reinvested into new use cases. When companies with exploding bills ask "where can we use more?" rather than "how do we cut this?", demand is becoming price-insensitive.
Agents push this one step further. Approving one agent workflow adds one always-on workload — unlike people, agents don't go home. The demand ceiling is being redefined from "users × working hours" to "agents × 24 hours."
From an infrastructure design perspective, training and inference demand different things. Training is a project-like, batch workload concentrated in a few power-rich sites; inference is user-proximate, geographically distributed, always-on (24×365), and bottlenecked not by compute but by memory bandwidth and latency. The bottleneck itself is also moving: over 60% of 2026 hyperscaler capex goes to non-chip items — power, cooling, buildings — and the binding constraint on new regions has shifted from GPU procurement to grid interconnection. The biggest change in the field over the past two years: the first question in large-scale adoption discussions moved from "do you have GPUs?" to "when does the power come online?"
3. The Structural Shift in Semiconductor Demand — "AI Buys Bandwidth"
With inference now roughly two-thirds of AI compute, the bottleneck has moved from FLOPS to memory. The mechanism: during inference, LLMs keep intermediate computations of prior context resident in GPU memory as KV cache. Longer contexts and more concurrent users grow that cache linearly; reasoning models cache their "thinking tokens" too; and agent sessions persist for days, making memory occupancy permanent. If the training era's bottleneck was GPU core count, the inference era's bottleneck is HBM capacity.
Custom silicon is eroding the GPU monopoly, but the key point for semiconductor demand lies elsewhere: however fiercely GPUs and custom chips compete, HBM, advanced packaging, and high-speed networking are demanded by all of them. "Whoever wins, memory gets sold." For Korean memory makers, intensified chip competition broadens the demand base. And when the world's largest memory buyer publicly raises its capex guidance from $200B to $220B citing memory prices, that is official confirmation of a supplier's market — buyers accepting prices.
Consistency with Korean export data — but the surge is mostly price
In July 2026, Korean memory exports rose 276.9% while system semiconductors fell 0.7% — directionally consistent with "AI buys bandwidth." But one distinction matters greatly from a central bank's perspective: decompose the surge and it is mostly a price effect. DRAM export prices rose 270.3% (Bank of Korea export price index); volume contribution is estimated in the single digits to low teens. The terms-of-trade tailwind is real, but if prices reverse, export values fall back even with volumes unchanged. The key question for the export outlook is therefore not "are volumes growing?" but "how long do prices hold?" — and the case for persistence rests on HBM's multi-year pre-order contract structure.
| Risk | Early-warning indicator | Status (Aug 2026) |
|---|---|---|
| Demand slowdown | Hyperscaler RPO growth rate | Accelerating — no alarm |
| Financial limits | Capex / operating cash flow | 99% — near threshold; external funding expanding |
| Delayed recovery | AI services revenue growth | Triple-digit growth continues |
| Shakeout | Agent project cancellation rate | Gartner: 40%+ cancellations projected by 2027 |
| Oversupply | China CXMT capacity, new fab ramps | The biggest variable for 2028 supply-demand |
Sources: Gartner (2026), company disclosures, author's synthesis. The projected 40% cancellation rate is a dot-com-style shakeout, not an AI retreat — the infrastructure demand base (inference and agent usage) grows regardless.
The risks are equally clear: product concentration (the extreme memory-versus-system divergence), customer concentration (the real buyers are four or five hyperscalers — a single company's capex guidance has become a national export variable), and a possible supply-demand reversal in 2028, if massive new capacity ramps coincide with slowing demand growth. 2028, not 2027, is the real test — and whether multi-year pre-order contracts act as a buffer is the question.
4. The Next Axis of Demand — Physical AI and Edge Memory
2026 was the year "the ChatGPT moment for robotics" was declared. Robots are shifting from per-task programming to learning general capabilities and specializing (generalist→specialist). What is telling: in Korea, Physical AI inquiries come first not from robotics companies but from manufacturing engineering and quality departments at manufacturers. The source of demand is different.
For semiconductor demand, Physical AI has a three-layer compute structure — cloud (foundation model training: GPU + HBM), simulation (digital twins and synthetic data: GPU + graphics memory), and edge/device (real-time inference and control: edge SoC + LPDDR and low-power memory). If data center HBM is the first wave, edge memory is the second. A robot or vehicle carries several to dozens of times a smartphone's memory, and the volume base differs: millions of data center servers versus a potential market of hundreds of millions of robots and vehicles. Volume arrives later, though — meaningfully after 2028, which also makes it a candidate buffer against the 2028 reversal risk.
Korea sits on both sides of this cycle: as a supplier (edge memory broadening the demand base) and as an adopter (world-leading robot density plus the demographics that make automation most urgent). Physical AI is where AI diffusion meets manufacturing competitiveness — and because it deploys on-site, its domestic investment and employment effects may exceed those of data center AI.
Conclusion — Four Key Messages
Nature of the investment: a demand-pulled cycle
Signed contracts exceed supply capacity. But internal cash has reached its limits and external funding is expanding — watch the funding structure and credit channels.
Structural shift: rising memory intensity
Training→inference→agents compresses to "AI buys bandwidth." A structural tailwind for Korean memory — consistent with the July export data, though most of the surge is a price effect, and that distinction matters.
Export outlook: memory-led strength likely through 2027
Manage the risks — product and customer concentration, China's CXMT capacity, and the 2028 reversal scenario — through leading indicators: RPO growth, capex guidance, HBM contract structure.
The next cycle: Physical AI as a second demand axis
Edge semiconductors as a new demand axis plus Korean manufacturing's adoption demand — Korea's dual opportunity as both supplier and adopter.
Need a talk, article, or advisory on this topic?
I deliver executive seminars for central banks, financial institutions, and enterprises, contribute to industry reports, and advise on technology strategy.
Get in Touch