Choose vLLM safetensors loading strategies for local disks and NFS

Choose a vLLM safetensors loading strategy from filesystem behavior and available RAM, not model quality. Lazy, eager, and prefetch alter loading behavior without changing weight contents. Compare them on one checkpoint while separating cold reads from warmed operating-system pages. Otherwise a fast repeat on NFS or NVMe can be mistaken for an intrinsic property of the method and incorrectly extrapolated to new nodes.

Compare the reading paths

The vLLM reference describes safetensors-load-strategy: lazy uses memory mapping, eager reads data up front with greater host-memory demand, and prefetch helps prepare pages. Check options in the installed release. Network storage can behave differently from a local disk, so no method wins by its name alone. A loading strategy is also different from quantization: changing how the same tensor file is read does not reduce weight precision or produce a different model artifact.

Define the timing boundary

Mark loading start, completed weight reads, subsequent runtime stages, and the first acceptance response. Safetensors supports tensor and slice access, but that does not make the entire startup a disk-only operation. Kernel preparation, allocation, and compilation can follow reading. Timing only an HTTP health response can misrepresent model readiness. Keep the same completion criterion for every variant and verify that the loaded model actually produces a correct response rather than merely accepting a network connection.

Test both storage conditions

Build a table containing local NVMe and a network-mounted directory, with the strategies under each. Record cold and repeated series, peak host RAM, read failures, and time to the specified acceptance state. This is an experiment design, not a published benchmark. Do not clear shared production filesystem caches merely to improve experimental control. Use an isolated node or label the uncontrolled condition honestly. Include concurrent readers where several workers start together; a single model load cannot describe a deployment-wide restart burst.

Follow the diagnostic branches

If eager reduces read waiting but exceeds the node’s RAM allowance, it is an unacceptable tradeoff there. If lazy starts quickly but later stalls on page access, examine I/O behavior rather than changing the context limit. If all strategies spend the same time waiting after reading, investigate another startup stage. Do not attribute every delay to NFS without verifying the actual mount, network condition, file availability, and checkpoint identity. These checks distinguish a loader decision from a broken or unexpectedly different storage path.

Preserve an operational decision

Save the model revision, file sizes, runtime version, storage type, and memory availability. State whether cold reads were compared with cold reads and repeat reads with repeat reads. Re-test after moving to another storage system instead of copying the parameter mechanically. The deliverable is a chosen strategy with a host-RAM budget, readiness criterion, and fallback choice. It does not replace artifact-integrity checks: every loader still needs a complete, consistent set of weights, indexes, and configuration, regardless of when the operating system fetches file pages into memory.

NVMe for model serving: where a faster disk changes results; GPU autoscaling: a free card is not a ready replica.