In this article
A synthetic generation checks a selected external path; separate its budget and result from health probes and user SLOs.
Select a specific external path
Liveness concerns process viability, readiness concerns accepting work, and a synthetic probe executes a bounded user scenario through an external endpoint. It might examine DNS, TLS, authentication, routing and completion of a short answer. One request does not cover every model and region simultaneously. The probe manifest therefore lists the endpoint, model, response mode and expected properties. Do not silently replace a heavy production workload with an overly simple request and then treat the result as evidence of that workload’s availability. Equally, a probe should not become large enough to materially change the queue and service it measures.
Set the budget before execution
Fix maximum input, output limit, attempt duration, frequency and total concurrent probes. The OpenRouter parameter documentation helps verify public generation limit semantics; a limit is not the exact amount of text ultimately returned. Account for model specific output and reasoning behavior instead of selecting an obviously unsuitable allowance. Retries share a total budget, otherwise one failed check can produce a chain of additional generations. When the preceding attempt remains active, skip the next one or classify it separately according to an explicit rule. A budget related skip must remain visible rather than becoming a successful observation.
Check structure and completion
Criteria might include an allowed HTTP response, valid envelope, expected content and a terminal signal. For streaming, check body completion and application completion separately rather than stopping at the first frame. Literal answer equality is usually fragile; choose a verifiable property appropriate to the scenario instead. Prometheus practices usefully separate attempt counts, errors and duration. Keep authorization secrets out of results and use synthetic payloads. Preserve the original diagnostic error with safe data redaction. An undifferentiated failed flag makes it difficult to distinguish network failure, invalid credentials and a model that is not ready.
Do not substitute for the user SLO
The Google SRE guide to implementing SLOs connects indicators to defined events and denominators. Synthetic sampling is a separate observation source: its frequency, request lengths and routes differ from user traffic. A successful short generation does not establish availability for long contexts or tool calling. If user error rates rise while the probe stays green, examine scenario coverage and traffic segments rather than dismissing the alert. If only the probe fails, investigate its credentials, environment and limits. This branch preserves the check’s usefulness without treating it as a complete picture of service quality under actual load.
Preserve a compact result
A result record contains scenario identifier, versions, failure stage, duration, observed usage and the reason retries stopped. See vLLM health and readiness and the pilot checklist. Run generation on a separate schedule and let metric collection read the stored result; a heavy request on every scrape can create its own load. Conclusions are limited to the selected scenario and observation window. Review the check after changes to routing, model or schema. A stable green chart is meaningful only when each sample’s external contract coverage and remaining blind spots are clear.