Meta Turns Idle Inference Capacity Into Better Ads

Executive Summary

• Meta says it built a system that turns idle inference capacity in its fleet into additional ads-ranking and model-routing work, according to the company's engineering post.
• Two elasticity levers, Rank More Ads (RMA) and Serve Better Models (SBM), expand optional serving work across a shared pool of preemptible capacity without adding capacity orders, the post states.
• Baseline serving is protected through physical isolation for SBM, logical isolation for RMA, separate control loops, and millisecond-scale load shedding, per the same source.

Key Takeaways

• Meta frames an industry-wide "capacity paradox": inference fleets can be globally constrained while pockets of capacity sit locally idle because of peak provisioning, model-timing gaps, regional resource-ratio mismatches and accelerator-type mismatches.
• RMA raises the number of candidates evaluated by ranking stages and consumes both CPU and accelerator surplus; SBM shifts a share of requests to a more powerful model and consumes accelerator-only surplus, so the two levers together address a wider resource mix than either alone.
• When opportunistic capacity contracts, the system removes optional work rather than letting it spill onto capacity reserved for baseline serving, keeping baseline traffic within its latency and reliability targets.
• Meta says it is extending both levers' operating range, developing precomputation elasticity, expanding cross-region inference and model portability, and building an optimization-based allocator for capacity across levers.

Meta Engineering's Capacity Paradox and the Pool Behind It

Meta's engineering team describes a recurring mismatch in large inference fleets: capacity must be provisioned for peak demand under strict latency and reliability targets, yet demand and supply rarely align across time, regions, resource combinations and accelerator types. The post lists four causes. Peak provisioning leaves capacity temporarily idle in lower-demand periods. Inference capacity arrives on supply-chain timelines while models launch on separate schedules, creating a gap between delivery and consumption. Regional resource ratios can strand one resource, such as CPUs, while another, such as accelerators, limits serving. And fleets spanning multiple accelerator vendors, generations and architectures cannot always move workloads across hardware, stranding capacity on hardware that cannot immediately serve available demand.

Meta's answer is to aggregate instantaneous idle capacity into a shared opportunistic pool. That pool is preemptible, volatile and carries no long-term guarantees, so Meta argues only workloads with specific structural properties fit: high elasticity to add or remove large amounts of optional work, sustained quality improvement across the range the pool can supply, and limited cross-service dependencies. Ads recommendation, already a large consumer of dedicated capacity, is positioned as meeting those criteria.

Meta Engineering's Two Levers: RMA and SBM Explained

RMA targets candidate volume. The ads system runs a sequence of ranking stages in which increasingly powerful models evaluate progressively smaller candidate sets, selecting a small number of ads from millions of candidates in under a second. RMA increases the number of candidates evaluated when opportunistic capacity is available, raising the chance of finding relevant ads. Because those candidates pass through CPU-heavy retrieval and feature processing as well as accelerator-heavy inference, RMA generally needs both CPU and accelerator surplus.

SBM targets model choice. Ads evaluation can use models of differing complexity, and Meta notes it is inefficient to run the more powerful model for every request on dedicated capacity. SBM changes the share of requests evaluated by the more powerful model as accelerator capacity changes, holding the number of ads entering ranking stages constant so upstream CPU demand stays comparatively stable. A model router uses request-value signals to prioritize requests expected to benefit most. Meta says it periodically reallocates between the two levers based on measured performance and resource requirements.

Meta Engineering's Isolation and Control Strategy for Baseline Traffic

Meta's stated design requirement is narrow and strict: losing opportunistic capacity may reduce incremental ranking work, but it must not overload capacity serving baseline workload. The company enforces this differently per lever. SBM uses physical isolation, with baseline and more powerful models on separate routing targets; dedicated replicas are sized for the baseline model, and replicas on opportunistic capacity serve requests selected for the more powerful one. When that capacity contracts, the system reduces the share of requests routed to the more powerful model, and those requests fall back to the baseline model, whose capacity is already provisioned.

Related: Google Cloud Launches Gemini Enterprise for Legal

RMA uses logical isolation. The system calculates how much additional candidate-ranking work the replicas on opportunistic capacity can support and sets a global budget for RMA; individual requests need not be physically separated. Control runs through two workload-specific feedback loops. RMA uses a central proportional-integral-derivative controller plus client-side enforcement, adjusting candidate volume using observed tail latency as the signal. SBM uses a central controller for model coverage plus a client-side router that segments requests by predicted engagement likelihood, runs explore-and-exploit measurement of incremental benefit per segment, and keeps a calibration loop to hold realized traffic share near target as traffic mix shifts.

Meta Engineering's Latency Gap and Value-Aware Load Shedding

Central controllers operate on the order of minutes, so a sudden preemption can remove capacity before clients receive new targets. Meta bridges that interval with two layers of protection. Server-side shedding removes lower-value opportunistic work immediately when the pool loses capacity before clients' views catch up: for RMA, reducing candidates processed within individual requests; for SBM, applying an equivalent value-aware policy to requests routed to the more powerful model. Client-side shedding has RMA and SBM clients observe server status directly and shed lower-value opportunistic work when the server is overloaded, routing fewer additional ads or fewer requests to the more powerful model. Both react within a few milliseconds and use value signals when choosing what to remove, which Meta says minimizes the effect during the transition.

For deeper context, see our Agentic AI analysis: "Balyasny Deploys Gemini Models Across 200 Investment Teams".

Meta Engineering's Roadmap for Elasticity and Allocation

Meta lists three areas of current work. It aims to increase workload elasticity by extending RMA's candidate volumes across ranking stages and using more powerful models under SBM, and by developing new levers such as precomputation elasticity, which performs work ahead of request time and is therefore less constrained by the location and timing of live traffic. It is expanding the usable capacity pool through model portability and cross-region inference so that compatible models and supporting resources line up with available capacity. And it is building an optimization-based system to automatically allocate inference capacity across elasticity levers based on benefits to users and advertisers, subject to regional constraints, hardware compatibility and durability requirements for idle capacity. These are described as in-progress efforts, not delivered results.

The source does not name regions, data-center locations, accelerator vendors or advertiser metrics, so those rows and columns are left unsupported rather than estimated.

Additional coverage: Meta Muse Glimmer Launches as Zuckerberg Calls for Lower Open-Weight AI Barriers

What This Means for Practitioners

For infrastructure and ML platform teams, Meta's account puts the emphasis on workload shape rather than hardware procurement. The reported criteria, high elasticity, quality gains that hold across the extra capacity, and few cross-service dependencies, are a screening test for which services can safely consume preemptible capacity. The two-loop design, central controllers at minute scale plus client- and server-side shedding at millisecond scale, addresses the gap that typically makes volatile capacity unusable for latency-sensitive serving. The value-aware removal policy matters too: when capacity is reclaimed, the system drops the lowest-value optional work first, which preserves the metric a business actually cares about.

Meta Engineering Implementation Risks

Several limitations are visible in Meta's own description. Opportunistic capacity is preemptible without notice and carries no long-term guarantees, so the incremental quality depends on a supply the operator cannot contract for. The central feedback loops converge on the order of minutes, which is slower than a sudden preemption, leaving local shedding as the only immediate defense. Meta also notes that cross-region and cross-accelerator fragmentation limits how much idle capacity can actually be used, since capacity is useful only when compatible models and supporting resources are available in the same place. The allocation optimizer, precomputation elasticity and expanded portability are described as current work, not measured outcomes.

Editorial independence disclosure: this article is based solely on the supplied source and was written without input from the company named. Source note: Meta Engineering, published October 12, 2026.