Validate vLLM automatic and explicitly pinned attention backends

Accept a vLLM attention backend through its effective selection and support for the model configuration. Automatic selection follows compatibility rules, while an explicit override must pass validation. A backend name from another benchmark is not a recommendation for every GPU. Preserve dtype, head dimensions, and required features before confirming the execution path. Treat quantized GEMM and CUDA Graph mode separately: they interact but are not one switch.

Understand automatic selection

The attention-backend documentation describes priority traversal and configuration validation. Without an explicit choice, the runtime selects a compatible implementation according to its rules. An engine upgrade can therefore change selection even when launch arguments stay identical. Do not infer the backend only from GPU architecture or installed libraries. Preserve effective backend and engine version so that a changed computation path is not misreported as a model regression or attributed to hardware that never changed in the experiment.

Interpret explicit validation

Manual selection is checked against dtype, dimensions, and other constraints. A documented incompatibility reason is useful evidence. Do not bypass it through random simultaneous changes. Consult the support matrix before testing another path and repeat functional acceptance. Successful startup does not establish support for every application feature. Additional modes can impose their own restrictions, so list requirements before tuning rather than discovering them only after a configuration appears to work. Keep intended functionality separate from one successful basic request.

Build an acceptance table

For a fixed model and GPU, create rows for automatic selection, the intended explicit backend, and an unsuitable choice used only to validate rejection. Record requested choice, effective choice, validation message, and functional fixture. This is an observation template, not a speed ranking. Compare paths after upgrading with identical inputs. If automatic selection changes, identify the supporting rule. If a manual path is rejected, retain its precise reason instead of describing the GPU generically as insufficiently powerful without reference to the actual compatibility check.

Validate graphs separately

The CUDA Graphs design relates capture modes to attention capabilities. A backend can support the model while requiring another graph path for certain shapes. Keep separate rows for attention selection and capture mode. FULL capture does not prove the desired attention kernel was selected, and the reverse is also untrue. Exercise mixed prefill/decode shapes occurring in the intended workload rather than accepting one short request that covers only a narrow branch. Preserve observed fallback or rejection as part of the result.

Record the selection decision

Benchmark latency and throughput only after functional acceptance, without promising explicit pinning always beats automatic selection. Pinning controls choice but can exclude future compatible paths. Preserve the reason and upgrade-review procedure. Include model, GPU capability, runtime, features, and effective backends in the release artifact. Revalidate on another node: equal VRAM and the same checkpoint do not establish identical architectural conditions, kernel support, or available attention execution. The useful conclusion explains what ran and why it was valid, not a universal quality claim attached to an implementation name.

Install vLLM: Check GPU, Driver and Package Compatibility; A reference GPU-node configuration for a small pool.