# HTTP/2 GOAWAY during inference endpoint updates

Test HTTP/2 GOAWAY during inference updates through last-stream-ID, accepted requests, ambiguous POST outcomes and a bounded draining experiment.

Author: Retrivium team
Updated: 2026-11-01
Canonical URL: https://retrivium.ai/journal/provider-http2-goaway-draining/
Language: en

One HTTP/2 connection can carry several streams. Closing it during rollout therefore cannot be treated as completion of exactly one inference request. GOAWAY communicates connection-processing boundaries, while each generation still needs application-level accounting. A bounded draining test should connect the transport signal, new-request admission and outcomes of existing responses.

## Separate connection from stream

[RFC 9113](https://www.rfc-editor.org/rfc/rfc9113.html) defines HTTP/2 streams and GOAWAY, including last-stream-ID. That identifier helps describe processing on one connection. It is not a model request ID and must not replace application generation identifiers in service records.

Where test infrastructure exposes it, preserve the mapping between client operations and transport streams. Do not infer every request's fate from one closed socket. Some operations may have completed, others may still be processing, while new work should no longer enter through the retiring path.

## Describe rollout behavior

[Kubernetes documents termination flow](https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/), where readiness and endpoint state participate in removing an instance from service. This differs from HTTP/2 GOAWAY. Having one mechanism does not establish that the other is configured or used on the relevant segment.

Before testing, diagram client, load balancer, API process and engine. Identify where HTTP/2 terminates and graceful shutdown occurs. If a proxy changes protocol, downstream signals may not describe its engine connection. Preserve that boundary in conclusions.

## Create a bounded experiment

Open a small number of synthetic streams with known durations and sizes. Initiate agreed termination of the test instance while they run. Send a separate short request after draining begins. Observe which operations are accepted, completed, rejected or left ambiguous.

Real customer traffic is unnecessary. Set an upper waiting limit and stop conditions to avoid indefinite connections. Save per-request timelines rather than only the total success count. Otherwise a rare truncation is hard to associate with a rollout phase.

## Do not promise safe POST replay

[RFC 9110](https://www.rfc-editor.org/rfc/rfc9110.html) distinguishes idempotent methods and cautions around retrying non-idempotent requests. A generation POST does not automatically become replay-safe because GOAWAY exists. Evidence of acceptance or a separately defined application policy is needed.

Inspect the selected library's behavior: whether it opens another connection, repeats a request and what it returns to callers. No visible error does not prove execution happened once. Preserve attempt count and its relationship to one logical operation.

## Preserve result boundaries

A successful draining test establishes behavior for the tested path and component versions, not universal resilience across load balancers or abrupt process failures. Test grace-period exhaustion during long generation separately.

Related guides cover [fallback routing](/journal/inference-routing-fallback/) and [model canaries](/journal/model-update-canary/). They describe application strategy, while the GOAWAY trace verifies transport boundaries without claiming universal external-platform support. Keep the trace, timeout settings and rollout trigger together so a later failure can be compared with the exact conditions that previously passed.