AI chips race: architectures evolve as demand and bottlenecks surge
From data center GPUs to on-device NPUs, AI chips are in a supercycle of investment and innovation. A new wave of architectures, memory technologies, and packaging is reshaping competition—and exposing constraints from power to supply chains.
The AI silicon supercycle begins
In the AI Chips sector, The AI chips market is in a sustained breakout, powered by generative models moving from experimentation to enterprise-scale deployments. Semiconductor revenue is set to climb sharply this year, with AI accelerators and memory-heavy platforms leading the charge. Industry-wide semiconductor revenue is projected to grow 17% in 2024, a momentum shift driven by high-performance compute and advanced memory, according to Gartner forecasts.
Nvidia’s quarterly performance underscores the trend. The company’s data center revenue has surged on the back of H100, H200, and related platforms, with total quarterly revenue crossing $30 billion and data center sales exceeding $26 billion, as reflected in recent company filings. Hyperscalers are expanding AI capacity across training and inference, while enterprises increasingly adopt copilots, code assistants, and domain-specific models—expanding the addressable market beyond frontier models.
Memory, packaging, and the unseen bottlenecks
As compute scales, bottlenecks have shifted to memory bandwidth and advanced packaging. High Bandwidth Memory (HBM) has become a critical enabler, with SK hynix, Samsung, and Micron racing to ramp HBM3E and the next iterations. Yields, capacity, and availability of HBM stacks now directly gate accelerator supply; the result is multi-quarter lead times and tighter allocation for buyers without long-term contracts. On the packaging front, 2.5D/3D technologies such as CoWoS and hybrid bonding have become strategic choke points, pushing foundries and OSATs to aggressively invest in capacity and throughput.
Policy is starting to meet physics. Funding from industrial programs and national strategies—including the U.S. CHIPS Act—has prioritized advanced packaging and memory alongside logic. That support aims to alleviate system-level bottlenecks, reduce geographic concentration risk, and encourage new supply chain participation. Near-term, however, the practical constraint remains: accelerators are increasingly limited by the speed at which HBM and packaging capacity can scale, not just by transistor counts.