Invited Seminar · Bank of Korea

Where Is AI Infrastructure Demand Heading?
— Enterprise AI Investment: Bubble or Structural Shift

Annual capital expenditure by the Big 4 hyperscalers has reached roughly 2% of US GDP. Is this investment sustainable, and how is the demand behind it reshaping the semiconductor market? Notes from someone who has watched the AI infrastructure cycle from both the supply side (cloud) and the demand side (enterprise).

Jihwan Woo · September 2026 · Reconstructed from an invited seminar at the Bank of Korea Research Department

← Home

About this article

This article reconstructs the content of "Enterprise AI Investment Trends and Shifting Infrastructure Demand — A Hyperscaler Perspective," presented at an invited expert seminar of the Bank of Korea Research Department in September 2026. All data is based on public sources (earnings releases, IR disclosures, industry research, as of August 2026). Views are the author's own and do not represent any affiliated organization.

Summary

The center of gravity in AI demand is shifting from building models to using them — and that shift is changing the very nature of semiconductor demand. Statistics and disclosures lag by quarters; enterprise contract and architecture decisions happen in the field six to twelve months earlier. This article organizes those leading signals around four questions: ① Is this investment a bubble or a structural shift? ② How are enterprises actually using AI? ③ What does the training-to-inference-to-agents transition mean for semiconductor demand? ④ Where is the next axis of demand?

$650–725BBig 4 hyperscaler capex for 2026 (+77%) — roughly 2.2–2.4% of US GDP
$496BAWS backlog (RPO) — up $132B in a single quarter
2/3Share of AI compute now going to inference — the bottleneck moves from compute to memory

1. Bubble or Structural Shift — Four Tests

Start with scale. Big 4 hyperscaler capex for 2026 is $650–725B, up roughly 77% year over year — around 2.2–2.4% of US GDP. Against historical infrastructure buildouts, that already exceeds the dot-com telecom ratio (~1.2%) but falls short of the railroad peak (~6%). Not unprecedented in scale, but unprecedented in speed. And there is one decisive difference: railroads and power grids depreciate over decades, while AI servers depreciate over five to six years. The verification cycle is short — the sustainability of this cycle depends on how quickly revenue recovery is proven.

Hence four tests.

Test 1

Backlog — contracts first, construction later

AWS's remaining performance obligations (RPO) stand at $496B, up $132B in a single quarter. RPO is signed-but-unrecognized revenue — a leading indicator for capex and semiconductor exports. Unlike the dot-com era, signed contracts now exceed supply capacity.

Test 2

The circularity critique — real, but not yet cycle-invalidating

Chip suppliers invest in model companies, which buy compute with that money — the circular structure is real. But three things differ from dot-com vendor financing: paying end demand (subscriptions, API revenue) actually exists and is growing fast; equity investments are separated from revenue recognition; and most of the backlog growth comes from ordinary enterprise customers. Circularity deserves monitoring, but it is not yet grounds to dismiss the cycle.

Test 3

Revenue recovery — is the investment coming back?

AWS maintains a 39.4% margin while disclosing $25B+ annualized AI revenue growing at triple digits. "AI and core cloud are pulling each other" is the supplier-side explanation — and so far the numbers support it.

Test 4

End demand — adoption passed the threshold, proof is halfway

74% of large enterprises have reached production deployment, escaping "pilot purgatory." Yet about half cannot measure ROI consistently, and 48% bolted AI on without redesigning workflows. The bottleneck is organizational, not technical.

The most honest demand indicator: how customer questions changed

YearTypical question in enterprise meetingsDemand stage
2023"Can we build something like ChatGPT too?"Curiosity & exploration
2024"Which model is best? What are the benchmark scores?"Model-centric thinking — PoC proliferation
2025"The PoC worked — now what about security review and cost control in production?"Transition to operations
2026"How much authority can we give an agent? Why is our token bill exploding?"Operating at scale

Generalized pattern from the author's field experience.

Model questions have nearly disappeared; only operations questions remain. Models themselves are becoming commodities, and differentiation has moved to data, workflows, and governance. The maturity of the questions is the maturity of the spending — exploratory one-off PoC budgets and operating expenses differ fundamentally in persistence, and that is the micro-level background of the backlog growth.

2. How Enterprises Actually Use AI — The Token Consumption Explosion

Enterprise infrastructure choices start with APIs and then differentiate by workload as usage grows — not a wholesale retreat but a portfolio: regulated data to private endpoints, predictable high-volume workloads to multi-year commitments, real-time UX to user-proximate deployment. The moment a company converts to a multi-year commitment is the moment its AI spending is reclassified from experiment to operating expense — the micro-level substance of that $496B backlog.

Meanwhile, consumption is exploding structurally. Unit prices are flat or falling, yet per-query consumption keeps rising, for four structural reasons: ① long context — feeding entire documents and codebases has become standard; ② retrieval (RAG) — every query carries dozens of pages of grounding documents; ③ agent loops — plan, call tools, review, retry: one task becomes dozens of model calls; ④ reasoning chains — models spend tens of thousands of tokens "thinking" before answering. One large health insurer's monthly token consumption grew from 3 million to 150 million in a year — 50x. 93% of enterprises have exceeded their AI budgets (McKinsey), and the cause is volume, not unit price.

Jevons paradox — unit price and consumption moving in opposite directions

Cut a customer's unit price by 30% and months later the total bill is higher — the saved budget gets reinvested into new use cases. When companies with exploding bills ask "where can we use more?" rather than "how do we cut this?", demand is becoming price-insensitive.

Agents push this one step further. Approving one agent workflow adds one always-on workload — unlike people, agents don't go home. The demand ceiling is being redefined from "users × working hours" to "agents × 24 hours."

From an infrastructure design perspective, training and inference demand different things. Training is a project-like, batch workload concentrated in a few power-rich sites; inference is user-proximate, geographically distributed, always-on (24×365), and bottlenecked not by compute but by memory bandwidth and latency. The bottleneck itself is also moving: over 60% of 2026 hyperscaler capex goes to non-chip items — power, cooling, buildings — and the binding constraint on new regions has shifted from GPU procurement to grid interconnection. The biggest change in the field over the past two years: the first question in large-scale adoption discussions moved from "do you have GPUs?" to "when does the power come online?"

3. The Structural Shift in Semiconductor Demand — "AI Buys Bandwidth"

With inference now roughly two-thirds of AI compute, the bottleneck has moved from FLOPS to memory. The mechanism: during inference, LLMs keep intermediate computations of prior context resident in GPU memory as KV cache. Longer contexts and more concurrent users grow that cache linearly; reasoning models cache their "thinking tokens" too; and agent sessions persist for days, making memory occupancy permanent. If the training era's bottleneck was GPU core count, the inference era's bottleneck is HBM capacity.

Custom silicon is eroding the GPU monopoly, but the key point for semiconductor demand lies elsewhere: however fiercely GPUs and custom chips compete, HBM, advanced packaging, and high-speed networking are demanded by all of them. "Whoever wins, memory gets sold." For Korean memory makers, intensified chip competition broadens the demand base. And when the world's largest memory buyer publicly raises its capex guidance from $200B to $220B citing memory prices, that is official confirmation of a supplier's market — buyers accepting prices.

Consistency with Korean export data — but the surge is mostly price

In July 2026, Korean memory exports rose 276.9% while system semiconductors fell 0.7% — directionally consistent with "AI buys bandwidth." But one distinction matters greatly from a central bank's perspective: decompose the surge and it is mostly a price effect. DRAM export prices rose 270.3% (Bank of Korea export price index); volume contribution is estimated in the single digits to low teens. The terms-of-trade tailwind is real, but if prices reverse, export values fall back even with volumes unchanged. The key question for the export outlook is therefore not "are volumes growing?" but "how long do prices hold?" — and the case for persistence rests on HBM's multi-year pre-order contract structure.

RiskEarly-warning indicatorStatus (Aug 2026)
Demand slowdownHyperscaler RPO growth rateAccelerating — no alarm
Financial limitsCapex / operating cash flow99% — near threshold; external funding expanding
Delayed recoveryAI services revenue growthTriple-digit growth continues
ShakeoutAgent project cancellation rateGartner: 40%+ cancellations projected by 2027
OversupplyChina CXMT capacity, new fab rampsThe biggest variable for 2028 supply-demand

Sources: Gartner (2026), company disclosures, author's synthesis. The projected 40% cancellation rate is a dot-com-style shakeout, not an AI retreat — the infrastructure demand base (inference and agent usage) grows regardless.

The risks are equally clear: product concentration (the extreme memory-versus-system divergence), customer concentration (the real buyers are four or five hyperscalers — a single company's capex guidance has become a national export variable), and a possible supply-demand reversal in 2028, if massive new capacity ramps coincide with slowing demand growth. 2028, not 2027, is the real test — and whether multi-year pre-order contracts act as a buffer is the question.

4. The Next Axis of Demand — Physical AI and Edge Memory

2026 was the year "the ChatGPT moment for robotics" was declared. Robots are shifting from per-task programming to learning general capabilities and specializing (generalist→specialist). What is telling: in Korea, Physical AI inquiries come first not from robotics companies but from manufacturing engineering and quality departments at manufacturers. The source of demand is different.

For semiconductor demand, Physical AI has a three-layer compute structure — cloud (foundation model training: GPU + HBM), simulation (digital twins and synthetic data: GPU + graphics memory), and edge/device (real-time inference and control: edge SoC + LPDDR and low-power memory). If data center HBM is the first wave, edge memory is the second. A robot or vehicle carries several to dozens of times a smartphone's memory, and the volume base differs: millions of data center servers versus a potential market of hundreds of millions of robots and vehicles. Volume arrives later, though — meaningfully after 2028, which also makes it a candidate buffer against the 2028 reversal risk.

Korea sits on both sides of this cycle: as a supplier (edge memory broadening the demand base) and as an adopter (world-leading robot density plus the demographics that make automation most urgent). Physical AI is where AI diffusion meets manufacturing competitiveness — and because it deploys on-site, its domestic investment and employment effects may exceed those of data center AI.

Conclusion — Four Key Messages

01

Nature of the investment: a demand-pulled cycle

Signed contracts exceed supply capacity. But internal cash has reached its limits and external funding is expanding — watch the funding structure and credit channels.

02

Structural shift: rising memory intensity

Training→inference→agents compresses to "AI buys bandwidth." A structural tailwind for Korean memory — consistent with the July export data, though most of the surge is a price effect, and that distinction matters.

03

Export outlook: memory-led strength likely through 2027

Manage the risks — product and customer concentration, China's CXMT capacity, and the 2028 reversal scenario — through leading indicators: RPO growth, capex guidance, HBM contract structure.

04

The next cycle: Physical AI as a second demand axis

Edge semiconductors as a new demand axis plus Korean manufacturing's adoption demand — Korea's dual opportunity as both supplier and adopter.

Key sources — Amazon Q2 2026 Earnings & IR (backlog, AI revenue, capex guidance) · NVIDIA FY26–FY27 earnings · IEA Energy and AI (2026) · TrendForce, Counterpoint Research (DRAM/HBM market) · Gartner, Deloitte, McKinsey, Forbes (enterprise AI adoption and ROI surveys, 2026) · Korea Ministry of Trade, Industry and Energy export statistics (July 2026) · Bank of Korea export price index (July 2026). Market data as of August 2026, based on public sources. Views are the author's own and do not represent any affiliated organization.
AI Infrastructure Hyperscaler Capex AI Bubble HBM Semiconductor Exports Inference Infrastructure Agentic AI Physical AI Bank of Korea Seminar

Need a talk, article, or advisory on this topic?

I deliver executive seminars for central banks, financial institutions, and enterprises, contribute to industry reports, and advise on technology strategy.

Get in Touch