# SGLang memory fraction: diagnose GPU memory pressure

Diagnose SGLang memory pressure using mem-fraction-static, prefill and decode stages, cache capacity and small controlled configuration changes.

Author: Retrivium team
Updated: 2026-11-01
Canonical URL: https://retrivium.ai/journal/sglang-memory-fraction/
Language: en

In SGLang, `--mem-fraction-static` controls the memory fraction for static allocations associated with model weights and the KV-cache pool. It is not a universal switch for fitting any workload. A change can create room for other components while reducing useful cache capacity.

## Break memory use into components

Weights, KV cache, activations and computational buffers all contribute to demand. The model file size therefore does not describe the whole runtime requirement. Read the tuning guidance for the installed version and identify the stage where allocation fails.

Record free device memory and neighbouring processes before each test. A percentage from another machine can have a different effect when the device, model or available memory changes. Do not compare results without preserving those starting conditions.

## Use a reproducible argument example

For an already prepared compatible environment, the source illustrates:

```bash
python -m sglang.launch_server \
  --model-path Qwen/Qwen3-0.6B \
  --host 127.0.0.1 --port 30000 \
  --mem-fraction-static 0.70
```

The `0.70` value demonstrates syntax, not a recommended universal configuration. Check the server-argument reference for your installed release and preserve the complete command. A working small model is a control, not proof that a larger target will fit with the same settings.

## Separate input and generation pressure

Long input processing involves prefill work and its buffers. Growing sequences and concurrency create another demand profile during generation. Save the log and workload at the failure point instead of categorising every OOM as the same shortage.

Use three stages: one short request, one long request and several typical requests together. Keep model and input examples fixed. This identifies which operating dimension changes the boundary and avoids masking the original cause with several simultaneous adjustments.

## Compare margin with useful capacity

Test two or three nearby values and record allocation, cache-related behaviour, latency and successful completion. Reducing the static fraction may relieve one pressure while reducing the number of active tokens the engine can hold. Removing an error is only part of the acceptance result.

Repeat the selected value from a fresh session and inspect the representative workload again. Preserve the last working profile and its limits. The useful outcome is a documented combination of model, context, concurrency and memory budget, which can then inform whether the next step is tuning, another replica or more device memory.

## Turn the check into a practical next step

A memory fraction is useful only within a measured model, context and concurrency profile. Preserve operating margin as well as cache capacity.

For the documented details, consult the [primary documentation](https://docs.sglang.io/docs/advanced_features/hyperparameter_tuning); [configuration reference](https://docs.sglang.io/docs/advanced_features/server_arguments). Continue with [sglang docker launch](/journal/sglang-docker-launch/) and [vllm kv cache preemption](/journal/vllm-kv-cache-preemption/).