In this article
When SGLang uses a checkpoint engine, distinguish process readiness from receiving actual weights.
When SGLang uses a checkpoint engine, distinguish process readiness from receiving actual weights. A dummy initial load can prepare the runtime before checkpoint delivery, but it does not establish valid inference. Define a gate that opens after the necessary loading and functional checks. A conventional health probe can otherwise permit production requests before the selected loading workflow is complete.
Interpret dummy initialization
The checkpoint-engine guide describes weight transfer and named checkpoints. An initial placeholder state belongs to a particular workflow and should not be interpreted as a loaded public model. Preserve the expected weight source, checkpoint name, and revision. A local model directory alone does not identify what reached the runtime. Where a process accepts weight updates, process startup and arrival of the intended parameters are different events that need separate observation and documentation.
Select the readiness gate
The server-argument reference includes checkpoint-engine-wait-weights-before-ready for waiting on initial weights. Check exact support and behavior in the installed version. This setting addresses a startup boundary, not the entire model-release policy. Unlike a general canary strategy, the first question here is whether the instance received its intended checkpoint at all. Preserve the condition in orchestration configuration so that another operator does not accidentally restore an earlier probe designed for ordinary immutable-weight loading.
Exercise the state sequence
Define runtime prepared, weights pending, named checkpoint received, readiness permitted, and acceptance request passed. Associate an observation source and timeout with each. This is an integration-test design, not a claim about deployed Retrivium tooling. The test controller should withhold ordinary inference until the required checkpoint arrives. Include delayed delivery as a negative case: the gate should remain closed rather than succeed because the process has stayed alive and answered a network probe for a long time without throwing an exception.
Confirm loading identity
Match delivery metadata with the expected checkpoint and receiving-process logs, then run a saved functional fixture. A plausible response does not prove exact equality of all tensor bytes. Conversely, checksums of external files do not prove those weights were applied inside the GPU runtime. Combine these distinct pieces of evidence. If the named checkpoint differs, fail acceptance for the intended version instead of explaining unexpected responses through sampling or silently changing the expected fixture to match whatever the server returned.
Separate startup from later updates
Initial weight waiting and subsequent parameter replacement have different state transitions. Do not extend an initial-readiness guarantee automatically to every update. A later-update workflow needs its own rule for when requests are allowed and which checkpoint they use. Keep versions, metadata, event sequence, and rollback source. The deliverable is a reproducible gate with intelligible closed-state reasons: pending delivery, wrong artifact, or failed functional acceptance. This prevents a successfully initialized dummy state from being mistaken for a production model merely because the surrounding runtime appears healthy.
Related reading
Updating an LLM on a GPU: testing a new version with a small rollout; vLLM is running but not ready: health checks and the first request.