In this article
Alert thresholds follow the SLO definition and budget-consumption rate. They are not universal constants for every inference service.
A brief error burst and gradual degradation need different responses. A simple error-percentage alert can miss sustained budget consumption or fire repeatedly at low traffic. Burn rate relates observed bad events to an allowed budget. For inference APIs, that arithmetic matters only after defining successful streams and denominator membership.
Begin with the SLO definition
Google SRE’s alerting guidance treats alerts as detection of significant error-budget consumption. It does not prescribe a mandatory OpenRouter or universal inference SLA. Products choose objectives and periods, then express measurable good and total events.
Keep the SLI definition beside the rule. If midstream failures are bad events, count them despite HTTP 200. If cancellations are separate, classify them before calculating rates. An attractive threshold cannot repair incorrect outcome accounting.
Explain burn-rate meaning
Burn rate compares the observed bad-event fraction with the fraction allowed by the objective. The dimensionless ratio describes consumption pace rather than guaranteeing it will persist. A short window is an observation, not a month-long reliability forecast.
The SLO implementation guide separates measurement from acceptable-reliability decisions. Document event units, windows, missing-data behavior and action ownership. A metric’s existence does not itself determine when a person needs urgent interruption.
Use multiple windows
A short window detects rapid spending while a longer one supports evidence of persistence. Combining them reduces some noisy behavior, but values need testing against the selected traffic. Do not copy example numbers as universal settings without explaining objectives and response time.
Streaming requests introduce a delay between start and completion. Preserve completion-based counter semantics where selected and examine long operations. An apparently late alert may result from the measurement boundary rather than faulty arithmetic.
Check low traffic
With few requests, one failure changes the fraction sharply. No requests and no observations are different states. Define when to produce diagnostic information, require another signal or withhold an urgent alert because evidence is insufficient.
Synthetic inputs should include quiet periods, short bursts, persistent degradation and recovery. Give each expected firing and clearing times. These are fixtures, not historical service data or evidence of actual project reliability.
Test rules before enabling them
Prometheus promtool supports rule checks against supplied time series. Save inputs, expected alerts and expression versions. Repeat fixtures after denominator or metric-name changes rather than inspecting dashboard appearance alone. Label mistakes can disable an alert without a syntax error.
Related guides cover GPU monitoring and capacity planning. The deliverable is an explainable response rule for observed degradation, not a universal SLA, a zero-error promise or a claim that a company already operates this alerting system. Preserve the test expectation for recovery too, since an alert that detects failures but never clears can create a different operational burden from the one it was intended to reduce.