In this article
A live process, completed startup and request readiness are different states. A poorly chosen probe can repeatedly interrupt normal model loading.
A large model can take longer to load than a conventional application. If liveness expects immediate generation, orchestration may restart the process before initialization finishes each time. CPU and disk stay busy while the service never becomes available. The opposite mistake is readiness based only on an open port, routing work to an unprepared model. Three probes can separate these states when configured appropriately.
Define each probe’s purpose
Kubernetes documents startup, readiness and liveness separately. Startup protects lengthy initialization, readiness governs traffic eligibility, and liveness helps detect conditions requiring restart. Replacing these roles with one heavy request requires careful justification.
First define the action following failure. If the model temporarily cannot accept work, should the process restart? If it is stuck, is traffic removal sufficient? Different answers need different signals even when both use HTTP probes.
Measure actual startup
Record container launch through readiness of the selected model and interface. Use several bounded observations under normal site conditions. One successful warm start does not describe initial file loading or recovery after an environment change.
The Pod lifecycle reference connects container state, probes and service handling. Settings should reflect measurements without allowing indefinite waiting. Preserve thresholds and intervals with explanations of the protected stage and the test’s stopping point.
Do not confuse ports with model readiness
The vLLM online-serving documentation lists interfaces and auxiliary endpoints. Check the installed version. A health route does not establish that an arbitrary check confirms the required model, tokenizer and launch settings.
Readiness should reflect an explicit application criterion. A lightweight state check can be paired with a separate bounded synthetic request after startup. Long generation on every scrape can itself become recurring load and affect service queues.
Exercise failure states
One fixture delays initialization while keeping the process alive. Another models temporary unavailability for new work. A third checks the selected restart-worthy condition. These belong in a test environment rather than an instruction to disrupt someone else’s production machine.
For each, preserve probe result and orchestration action. If normal loading is repeatedly restarted, correct signal meaning or startup budget rather than disable all checks. If traffic arrives too early, investigate readiness and routing independently of liveness.
Bound conclusions to configuration
Passing establishes consistency between criteria and the selected application version, not generation quality or freedom from every possible hang. Repeat startup traces after model or loading changes because the former timing budget may no longer fit.
Related guides cover vLLM running but not ready and GPU cold starts. They explain symptoms and preparation time; the probe contract defines safe automated responses to each observed server state. Retain failed-fixture evidence as well as successes, since a probe that always returns healthy cannot prove it would protect a real rollout.