In this article
With SGLang prefill-decode disaggregation, separate KV-transfer delay from computation on both stages.
With SGLang prefill-decode disaggregation, separate KV-transfer delay from computation on both stages. A fast prefill does not guarantee a fast first output token when state moves slowly between workers. Build a request timeline and verify the selected transport. Increasing decode-GPU capability without locating the wait may change another stage while leaving the actual restriction unresolved.
Map the request path
A prefill worker processes the input before the resulting state becomes available to a decode worker. The PD-disaggregation guide describes transport choices including Mooncake and NIXL and their requirements. This is more than running two HTTP servers with the same model. Check configuration agreement, worker identities, and transfer conditions before measuring. Matching model names alone does not establish compatible state representation or correct delivery through the selected implementation. Record the expected sender and receiver explicitly.
Verify hardware mapping
Preserve the relationship among GPUs, network devices, and processes. A fast physical interface does not prove that the transport uses the intended route. Inspect configuration and diagnostics rather than inferring the path from the network-card model. Distinguish intra-node from inter-node transfer because topology and constraints differ. Record required libraries and environment support before comparing results. An open port establishes connectivity at one boundary, not a working state-transfer data path or the hardware route actually carrying it.
Create a request timeline
For one request, record acceptance, prefill completion, state-transfer start and completion, decode start, and first client response wherever those boundaries are observable. Mark unavailable timestamps instead of inventing precise transfer duration from an unexplained residual. Compare short and long inputs while keeping output controls fixed. If the interval between prefill completion and decode start grows, investigate transfer and receiver waiting rather than only the sender’s attention-computation speed. Preserve request identity with every observation so that stages cannot be accidentally combined across requests.
Profile workers separately
The official profiling guide calls for separate profiling in PD deployments. Correlate observations through request identity and aligned timelines. One GPU trace cannot describe the other worker, while average utilization across both can hide a transfer stall. Repeat the series with fixed placement and unchanged scheduler budgets. A busy receiver introduces waiting that must be distinguished from copying itself; otherwise subsequent optimization targets the wrong layer and a transport change may appear to fix or worsen an unrelated scheduling condition.
State a localized conclusion
Include topology, transport, versions, input lengths, and observed stage boundaries in the report. Distinguish direct measurements from indirect estimates. Test transfer failures separately without promising seamless recovery from every interrupted movement. Repeat functional and timing acceptance after switching transports. The deliverable is a reproducible request record and a localized bottleneck, not generic advice to buy faster GPUs or networking. Disaggregation helps only when the complete path meets workload requirements; it does not automatically remove KV-movement cost or establish superiority over colocated prefill and decode without a separate comparison.
Related reading
Prefill and decode: the work your GPU performs during inference; PCIe and GPU hosting: when the connection limits a model.