# Prefill/decode disaggregation (PD disaggregation)

> VI: https://5ac.vn/wiki/prefill-decode-disaggregation/ · EN: https://5ac.vn/en/wiki/prefill-decode-disaggregation/ · lang: en · spec_as_of 2026-10-21 (spec no formal spec) · cluster C17

Infrastructure architecture that runs the two LLM inference phases, prefill and decode, on separate GPU pools joined by a router and a KV-cache transfer path.

---
## Definition

An LLM request moves through two phases with opposing hardware appetites. Prefill reads the whole prompt in one pass and is compute-bound. Decode emits one token at a time, re-reading weights and the KV cache every step, so it tracks memory bandwidth. Prefill/decode disaggregation (PD disaggregation) puts the two phases on separate GPU pools — a prefill pool and a decode pool — joined by a router and a KV-cache transfer path. This entry covers pool splitting only: techniques handling requests inside a pool, such as chunked prefill and prefix caching, are defined in the [inference engine](/en/wiki/inference-engine/) entry.

## Mechanism

Mixed in one pool, the two workloads interfere: a large prefill batch occupies the hardware and stretches decode latency for every other request. The foundational paper is DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving (arXiv:2401.09670), which assigns the phases to different instance groups and optimizes goodput — requests completed within the committed latency targets — instead of raw tokens per second.

Production numbers from the inference stack DeepSeek opened on 20 October 2026: the smallest efficient unit of DeepInfer is 64 prefill GPUs plus 64 decode GPUs, costing roughly $633.60 to $1,423.36 per hour at Google Cloud B200 rental rates. Disaggregation, with the persistent cache and the load balancers, sits in the layer that stayed closed — anyone who wants it assembles three pieces: a router between pools, a KV transfer path, and admission control.

## Application context

This problem reaches only large infrastructure. The 5ac guidance: treat the KV store and P/D disaggregation as multi-quarter self-built projects when you own 128 or more Blackwell-generation GPUs or an Ascend cluster with a dedicated systems team. Open-source engines have the road ready — vLLM and SGLang both carry PD disaggregation in their online-serving stacks. For small businesses, the cheaper line is API capacity at cache-hit pricing; the provider runs the split pools.

## Sources fetched

- [DeepSeek opens its production inference stack](/en/blog/deepseek-opens-production-inference-stack/) — the 64+64 GPU floor, $633.60–$1,423.36 per hour, the closed disaggregation layer — accessed 2026-10-21
- [Local AI 2026: From an RTX 3090 to Enterprise On-Prem Infrastructure](/en/blog/local-ai-2026-from-rtx3090-to-enterprise/) — prefill and decode as two different hardware problems — accessed 2026-10-21
- <https://arxiv.org/abs/2401.09670> — DistServe paper on disaggregating prefill and decoding (v3) — verified via arXiv API 2026-10-21

## Related

- [Inference engine (vLLM)](/en/wiki/inference-engine/)
- [Local LLM deployment](/en/wiki/local-llm-deployment/)
- [AI token cost](/en/wiki/ai-token-cost/)