# Reconcile llama.cpp unified KV pools and per-slot context limits

Account for a llama.cpp shared KV pool separately from the context of one parallel slot.

Author: Retrivium team
Updated: 2026-11-01
Canonical URL: https://retrivium.ai/journal/llama-cpp-unified-kv-slot-limits/
Language: en

Account for a llama.cpp shared KV pool separately from the context of one parallel slot. Total pool capacity does not mean every concurrent request can consume all of it. Check `kv-unified-per-slot`, slot count, and effective runtime limits. Acceptance should validate both the individual request contract and simultaneous execution, not merely a successful startup with a large overall context value.

## Distinguish pool and slot

The [llama.cpp argument definitions](https://raw.githubusercontent.com/ggml-org/llama.cpp/master/common/arg.cpp) describe a per-slot context limit and shared-pool sizing under specific unspecified-setting conditions. `kv-unified` concerns a shared sequence buffer. It neither automatically combines separate GPU memories into one device nor describes model-weight size. Record parallel slots and explicitly configured context before applying the relevant sizing rule. Keep units and limiting scopes separate rather than reducing several different controls to one apparently simple capacity number. The chosen mode determines which interpretation is valid.

## Inspect effective limits

The [server implementation](https://raw.githubusercontent.com/ggml-org/llama.cpp/master/tools/server/server-context.cpp) checks individual slot state and context. Obtain effective limits through supported runtime observations and acceptance behavior, rather than dividing any context number by parallelism without checking the mode. Preserve the build version because new options can alter allocation rules. If an input is rejected, retain its error and slot information. An individual context boundary and overall resource exhaustion explain different failures and call for different configuration changes. The distinction matters even when both prevent a response.

## Create a capacity map

For an illustrative three-slot server, make columns for slot, individual limit, occupied context, and state. Three is the example size, not a parallelism recommendation. Record shared-pool configuration separately. Test one long request and a concurrent series with preserved token lengths. Success of the isolated request does not demonstrate joint capacity because the remaining slots did not make competing demands. A complete acceptance record must show the conditions under which requests are intended to coexist, including their output budgets.

## Follow the diagnostic branches

If a long request is rejected despite a free pool, inspect its individual boundary first. If requests succeed alone but fail together, investigate shared capacity and actual slot occupancy. Do not change weight precision without evidence that weights caused the failure. Preserve arrival ordering, input lengths, and output settings. Growing conversation history also consumes context, so one short standalone prompt cannot validate continued dialogue behavior. Include a history-bearing fixture where that is part of the promised request contract.

## Preserve both boundaries

Document KV mode, slot count, individual and shared limits, and fixtures for isolated and concurrent execution. Identify explicitly supplied values and those derived by the runtime. Repeat acceptance after fit or parallel changes even if total context remains unchanged. The resulting map should explain one request's allowed length and the conditions for competition without claiming that every slot owns the whole pool. It should also distinguish an individual-contract violation from temporary shared-capacity pressure instead of reducing both to generic advice to increase context or VRAM.

## Related reading

[Long-context LLMs: testing a server for document workloads](/journal/long-context-capacity/); [llama.cpp server: a local API from a GGUF model](/journal/llama-cpp-server-launch/).