What changed in the latest MLPerf release for AI accelerators?

The December MLPerf round added new submissions from next-wave accelerators and refreshed software stacks, focusing on LLM training and inference. Nvidia H200 maintained the lead on raw multi-node training, while AMD MI300X and Google’s Trillium (TPU v6) posted efficiency gains that closed the inference gap. Intel Gaudi 3 improved price-performance and scaling, and AWS introduced Trainium2 public data emphasizing cost-per-token. The headline shift is toward cluster-scale efficiency and energy use rather than peak single-node numbers.

How do Nvidia H200 and AMD MI300X compare on LLM workloads?

Vendor data and MLPerf submissions show H200 leading on multi-node training throughput, propelled by higher HBM capacity and compiler/runtime enhancements. MI300X narrowed the gap on inference tokens-per-second, especially for memory-bound LLMs where its HBM3 is advantageous. For buyers, H200 often wins on peak training speed and software depth, while MI300X competes strongly on tokens-per-dollar and energy efficiency for production inference, depending on model size and quantization strategies.

What does Google’s Trillium (TPU v6) contribute to efficiency?

Trillium targets energy-efficient inference and robust training scale, aided by Google’s network fabric and XLA compiler optimizations. In Google Cloud, customers see predictable throughput and cost profiles for LLMs and mixture-of-experts models, which simplifies operations. It’s appealing for teams prioritizing managed environments, consistent SLAs, and low variability over low-level stack control. Efficiency metrics in December’s data highlight Trillium’s strengths on production inference at scale.

Where does Intel Gaudi 3 fit versus cloud-native options like Trainium2?

Gaudi 3 suits cost-sensitive deployments seeking competitive performance at mid-scale with maturing frameworks and kernels. It may trail the top systems on absolute training throughput, but networking and memory architecture enable solid scaling. Trainium2, tightly integrated with AWS services, emphasizes cost-per-token and elasticity, attractive for variable inference workloads. Organizations should weigh workload portability, ecosystem support, and regional capacity when choosing between on-prem Gaudi 3 and cloud-native Trainium2.

What metrics should enterprises prioritize when selecting AI accelerators?

Prioritize tokens-per-joule, tokens-per-dollar, and cluster scaling efficiency at 256–1024 accelerators. Examine fine-tuning wall-clock times, failure recovery behavior, and job scheduling predictability under mixed workloads. Validate claims using MLPerf submissions and third-party tests against your specific model family and quantization level. Also factor in software ecosystem maturity and support commitments, as compiler and kernel improvements can shift effective performance by double-digit percentages over the lifecycle of a deployment.

MLPerf December Scores Reorder AI Silicon: Nvidia H200 Leads as AMD MI300X, Google Trillium Tighten Race

Fresh MLPerf results released in December redraw the AI accelerator leaderboard. Nvidia's H200 leads training throughput, while AMD's MI300X and Google's Trillium TPU close the gap on inference efficiency and scaling, with AWS Trainium2 entering the chart on price-performance.

Published: January 5, 2026 By Aisha Mohammed Category: AI Chips

MLPerf December Scores Reorder AI Silicon: Nvidia H200 Leads as AMD MI300X, Google Trillium Tighten Race

Executive Summary

MLCommons’ December MLPerf release highlights Nvidia H200 leadership in training throughput, with AMD MI300X and Google Trillium TPU narrowing the inference efficiency gap (MLCommons).
Vendor disclosures indicate 10–25% gains gen-over-gen across key LLM and vision tests, led by memory bandwidth and compiler optimizations (Nvidia, AMD, Google Cloud).
AWS brings Trainium2 into public benchmarks with competitive cost-per-token and strong scaling on 256–1024-accelerator clusters (AWS News Blog).
Analysts flag tokens-per-joule and cluster-scale efficiency as primary enterprise buying criteria for 2026 AI buildouts (IDC, Gartner).

Benchmark Season Reset: What December’s MLPerf Shows

MLCommons’ latest MLPerf results, released in December, provide the clearest snapshot yet of how next-wave accelerators stack up on training and inference for large language models and vision workloads. The submissions feature new silicon and updated software stacks, with a focus on end-to-end throughput, scaling efficiency, and energy use on standardized tasks (MLCommons).

Across the suite, Nvidia’s H200-class systems maintain a lead in raw training throughput on multi-node configurations, a result the company attributes to higher HBM capacity and compiler/runtime updates in its software stack. Vendor commentary points to double-digit percentage gains over prior rounds, particularly on LLM pretraining and reinforcement learning from human feedback (RLHF) stages (Nvidia, Reuters).

Who’s Closing the Gap: AMD, Google, Intel, and AWS

AMD’s MI300X shows marked improvement in inference performance, with submissions and partner data indicating a narrowing gap versus Nvidia on tokens-per-second for popular open-weight models, supported by expanded HBM3 memory and refinements in the ROCm stack. AMD also highlights competitive total cost of ownership (TCO) in large clusters where memory-bound workloads dominate (AMD Newsroom, Bloomberg).

...

Read the full article at AI BUSINESS 2.0 NEWS