Inside Meta Andromeda's Single-GPU Ads Retrieval Redesign

Executive Summary

• Meta describes a next iteration of its Andromeda ads retrieval engine that co-locates index matching, fused scoring and top-K selection on a single GPU server, reporting over one billion ads per second in production scoring every eligible ad (Meta engineering report).
• The post says a fused scoring kernel reached over 10 billion ads per second on a single GH200 (Meta engineering report).
• Meta attributes a 2.4% conversion increase on Facebook since 2025 to Andromeda, and says the 2026 work builds on the earlier launch rather than replacing it as a first launch (Meta engineering report).

Key Takeaways

• The architectural core is a retrieval mixture-of-experts design: specialized user experts, joint ad encoding with real-time serving context, and an MoE attention gating network that routes candidates to top-K experts.
• Meta says the fused kernel keeps intermediate tensors in registers and shared memory, avoiding HBM materialization that it identifies as the dominant bottleneck at billions of ads per second.
• Moving index matching onto the GPU is described as difficult because advertiser targeting logic spans many feature categories and must support real-time index updates.
• Every throughput and efficiency figure is Meta-reported and was not a substitute for independent benchmarks; kernel-level rates should not be read as full pipeline rates.

Meta Andromeda Model Architecture Shift

According to the post, the 2024 Andromeda retrieval models established a GPU-optimized deep neural network foundation and grew model capacity by four orders of magnitude. Meta says that paradigm now needs to move: with generative AI creative tools helping advertisers produce hundreds of millions of new ads, the candidate pool has expanded and optimizing for hardware capacity return on investment has become more critical.

The 2026 design applies mixture-of-experts ideas to retrieval. On the user side, a single user tower becomes an ensemble of specialized user towers as experts, drawing on sequential ad engagement events, user-cohort representations, LLM-derived semantic features and cross-surface co-engagement signals. On the candidate side, a joint encoding strategy incorporates real-time serving context. The interaction layer changes from dense neural networks into an MoE attention-based gating network that routes candidates across the most relevant user experts.

Meta System Co-Design and the Fused Kernel

Meta frames retrieval as fundamentally memory-bandwidth bound. At billions of ads per second, even a small intermediate tensor per ad multiplies into terabytes per second of high bandwidth memory traffic, and the post says this memory workload, not arithmetic, becomes the dominant bottleneck in multi-kernel pipelines.

The response is a single GPU kernel fusing model and system workloads per user-ad interaction. The post says the roughly 300x FLOPs reduction in the gating architecture created headroom that made full fusion feasible, compared with the 2024 launch where only operations within model execution were fused. The fused kernel executes embedding gathering, interaction and scoring in one pass, keeping intermediate results on-chip. Separately, the co-location strategy places index matching, scoring and top-K selection on the same GPU, with only a compact request payload sent to the server, which Meta says reduces network traffic by roughly 100x.

Meta Throughput Claims and Hardware Headroom

The post says the 2024 system evaluated at tens of millions of ads per second per GPU card, and that the combined single GPU pipeline now processes over one billion ads per second on a single GPU server while scoring every eligible ad across Facebook and Instagram. It describes this as a new throughput benchmark for ads retrieval-stage models, a 100x lift over the 2024 Andromeda system.

Meta and NVIDIA launched a joint collaboration in early 2026 to push components toward hardware limits. The optimized gating kernel improved 5.3x, from 2.4 billion to over 10 billion ads per second, reaching about 85% of peak HBM bandwidth. INT8 quantization added up to 2x throughput over optimized FP16 kernels with accuracy described as on par with 16-bit precision. Index matching improved over 2x to more than 100 billion ads per second. These are component figures; the end-to-end pipeline benchmark is described more cautiously as multiple times current production throughput.

Meta Implementation Risks

The editorial cautions here matter for readers. The headline 100x lift compares the new production pipeline with the 2024 system, while the fusion and co-location gains are explained through kernel and network measurements. Readers should keep the fused scoring kernel rate distinct from the full production retrieval pipeline, which also includes index matching, top-K selection and request handling. Our editorial recommendation is to treat each figure according to the layer it describes rather than as a single system-wide number.

The post also indicates remaining bottlenecks and effort. Index matching required redesigning the index structure for GPU-parallel execution while preserving full fidelity of production targeting semantics, and real-time index updates must still be supported. Meta says the collaboration identified data movement between pipeline components, not computation, as a significant fraction of end-to-end latency, and that further gains are expected as additional optimizations integrate into production. Our editorial recommendation is to regard the reported headroom as capacity the source intends to use for deeper interaction networks, richer features and additional retrieval pathways, not as measured production output.

Meta Platform Roadmap and Interpretation

Meta says it is extending optimization and co-design work to Grace Blackwell, focusing on platform design choices that enable different CPU-to-GPU compute ratios, which the work so far suggests is an important factor for Andromeda-like workloads. The post also says results confirm the co-designed pipeline is not yet at its ceiling.

All results, including the conversion increase and efficiency claims, are Meta-reported and were not a substitute for independent benchmarks. The 2024 launch on the NVIDIA GH200 Grace Hopper Superchip is the prior foundation; the October 28, 2026 post describes a next generation, not a first launch of Andromeda.

What This Means for Practitioners

For teams building recommendation or ads systems, the source's central lesson is architectural: Meta attributes its gains to designing model, system and hardware as one integrated stack rather than optimizing each in isolation. The post gives concrete patterns to study, including routing candidates through a gating network to a limited set of user experts, encoding ads with real-time serving context, and collapsing per-user request logic onto the same GPU as scoring to avoid network and memory bottlenecks. It also shows that moving targeting logic to the GPU required preserving production semantics. Our editorial recommendation is to evaluate these patterns against your own workload measurements, since the reported figures come from Meta's systems and were not independently verified.

Editorial Independence and Source Note

Our recommendations are editorial judgment, not independent product testing. All figures and claims are attributed to Meta's engineering post and are Meta-reported. Source: Meta engineering report.

Related reading (not used as source evidence): Related AI Chips coverage, Related AI Chips coverage, Related AI Chips coverage.