Cerebras Systems reported on November 17, 2026, that its production disaggregated inference deployment, which assigns prompt processing to AMD Instinct GPUs and response generation to CS-3 systems, delivered five times the throughput per megawatt of a CS-3-only configuration without reducing token-generation speed.
The result applies to Cerebras Inference Cloud. The measurement came from a shared GPT-OSS-120B endpoint on November 14, 2026, comparing throughput per megawatt before and after disaggregation was enabled and normalizing the aggregated configuration to a 1× baseline. Shifting prompt work to the GPUs leaves the CS-3 systems’ serving capacity available for generating responses, expanding the fast-inference capacity Cerebras can offer through its cloud.
The production disclosure follows Cerebras’ October 1, 2026 explanation of disaggregated inference, which reported a 5× gain in early tests with the same wafer-scale-engine footprint. The new announcement moves beyond that early result by supplying a production measurement and a deployment package for infrastructure partners.
Why Cerebras Separates Prefill and Decode
Each inference request has two principal stages. Prefill processes the input prompt and creates intermediate model state; decode uses that state to produce output tokens one at a time, feeding each new token back into the model. Because every decode step reads model data from memory again, memory bandwidth is central to interactive generation speed. Prefill instead performs more computation across the prompt in parallel, making it a better match for accelerators suited to that compute load.
Cerebras’ wafer-scale systems place model data in on-chip SRAM rather than relying primarily on off-chip high-bandwidth memory. Cerebras describes its wafer-scale chip as carrying tens of gigabytes of SRAM and tens of petabytes per second of memory bandwidth—350× the SRAM capacity and 5,400× the memory bandwidth of a typical GB300 GPU. That capacity allows more concurrent requests and model layers to remain directly on the chip for rapid decode.
The economic target is more customer-serving capacity and revenue potential from the same power budget in power-constrained data centers. Cerebras cites Muse, Copilot and Codex as examples of rapidly scaling agentic applications that require highly interactive responses, so the capacity increase must preserve generation speed to be useful for those workloads.
Deployment Architecture and Software
Infrastructure partners can use a Cerebras-provided reference architecture to co-locate their chosen prefill accelerators and Cerebras systems on a unified network. They then have two integration routes: an end-to-end software stack intended to shorten time to market, or a decode-only SDK for partners that want more control.
The full stack routes requests between prefill and decode pools and manages replica capacity and health. Its prefill engine can run open-source inference engines such as vLLM or SGLang, while Cerebras supplies the decode engine. KV Portal transfers intermediate model state between the two engines over RDMA, including the key-value caches that preserve request-specific context after prefill.
With the decode-only option, partners retain their own inference framework and prefill engine while using Cerebras’ decode engine and KV Portal. The SDK also includes a mock decode replica and a reference integration with an open-source disaggregated inference framework and prefill engine. Amazon Web Services, Fireworks AI, Gimlet Labs and General Compute are named as early-access partners, and Cerebras is expanding access to additional partners.
Capacity Target
Cerebras plans to deploy disaggregated inference broadly across hundreds of megawatts in its own inference cloud and partner deployments by the end of 2027. That is a deployment target, not capacity already operating.
The announcement therefore combines a measured cloud configuration with a software path for outside operators. The central scaling question is whether the throughput advantage measured on one shared endpoint can be reproduced across partners’ chosen accelerators, frameworks and deployment sizes as Cerebras pursues that broader rollout.
Credit: Source link


























