In this article
GOAWAY concerns connection handling and stream-processing boundaries. It does not guarantee transparent replay or resumption of every generation.
One HTTP/2 connection can carry several streams. Closing it during rollout therefore cannot be treated as completion of exactly one inference request. GOAWAY communicates connection-processing boundaries, while each generation still needs application-level accounting. A bounded draining test should connect the transport signal, new-request admission and outcomes of existing responses.
Separate connection from stream
RFC 9113 defines HTTP/2 streams and GOAWAY, including last-stream-ID. That identifier helps describe processing on one connection. It is not a model request ID and must not replace application generation identifiers in service records.
Where test infrastructure exposes it, preserve the mapping between client operations and transport streams. Do not infer every request’s fate from one closed socket. Some operations may have completed, others may still be processing, while new work should no longer enter through the retiring path.
Describe rollout behavior
Kubernetes documents termination flow, where readiness and endpoint state participate in removing an instance from service. This differs from HTTP/2 GOAWAY. Having one mechanism does not establish that the other is configured or used on the relevant segment.
Before testing, diagram client, load balancer, API process and engine. Identify where HTTP/2 terminates and graceful shutdown occurs. If a proxy changes protocol, downstream signals may not describe its engine connection. Preserve that boundary in conclusions.
Create a bounded experiment
Open a small number of synthetic streams with known durations and sizes. Initiate agreed termination of the test instance while they run. Send a separate short request after draining begins. Observe which operations are accepted, completed, rejected or left ambiguous.
Real customer traffic is unnecessary. Set an upper waiting limit and stop conditions to avoid indefinite connections. Save per-request timelines rather than only the total success count. Otherwise a rare truncation is hard to associate with a rollout phase.
Do not promise safe POST replay
RFC 9110 distinguishes idempotent methods and cautions around retrying non-idempotent requests. A generation POST does not automatically become replay-safe because GOAWAY exists. Evidence of acceptance or a separately defined application policy is needed.
Inspect the selected library’s behavior: whether it opens another connection, repeats a request and what it returns to callers. No visible error does not prove execution happened once. Preserve attempt count and its relationship to one logical operation.
Preserve result boundaries
A successful draining test establishes behavior for the tested path and component versions, not universal resilience across load balancers or abrupt process failures. Test grace-period exhaustion during long generation separately.
Related guides cover fallback routing and model canaries. They describe application strategy, while the GOAWAY trace verifies transport boundaries without claiming universal external-platform support. Keep the trace, timeout settings and rollout trigger together so a later failure can be compared with the exact conditions that previously passed.