In this article
The setting is not compute utilisation or hardware isolation; a memory percentage does not guarantee service performance.
vLLM’s --gpu-memory-utilization concerns the engine’s memory budget. It is not a target percentage of GPU compute activity and does not isolate neighbouring applications. A higher value may provide more room for cache while reducing device headroom.
Change the setting only after recording initial free VRAM and the actual symptom. More cache memory does not fix a slow disk or incorrect API address.
Separate memory from compute activity
A monitor can show high memory use while compute activity is low. The engine may prepare memory pools before requests arrive. That observation does not establish useful generation or customer demand.
Preserve model, engine version and measurement source. A configuration percentage and an observed device-memory value are different records, and neither alone describes an application’s response time.
Keep headroom for the selected workload
The following illustrates syntax only:
vllm serve Qwen/Qwen3-0.6B --max-model-len 4096 --gpu-memory-utilization 0.80The value 0.80 is not a universal recommendation. Test representative requests, longer input and concurrency after startup. When comparing budgets, change only this setting while preserving the artifact and other conditions.
| Observation | Question |
|---|---|
| Before startup | How much memory is actually available? |
| After loading | What did the runtime prepare? |
| During a request | Where does the peak occur? |
| Under concurrency | Does the tested profile remain stable? |
| After completion | What state is retained? |
Do not mistake multiple instances for isolation
Two servers on one card still use one physical device. Their individual settings do not create independent GPUs. Test them simultaneously if simultaneous use is intended.
Background applications, context changes and additional buffers can alter the combined footprint. Record initial free memory before every comparison so a changed neighbouring process is not mistaken for the effect of the parameter.
Match the change to the problem
Insufficient KV-cache capacity, startup OOM and slow responses are different symptoms. Use the official serve reference and optimisation guidance to connect observations with relevant controls. Keep errors and the last successful stage before modifying the environment.
Repeat a short control request after changes and after a normal restart. Preserve useful quality as well as memory and timing. The owner’s schedule also needs a defined release procedure if the card serves other work.
Those observations support a practical capacity discussion without treating a percentage as guaranteed throughput or financial performance.
Continue with CUDA Out of Memory with Concurrent Requests and vLLM max-model-len: Set a Meaningful Context Limit.