In this article
Inject failures at known stream boundaries; after content delivery, a new request cannot silently become the old stream continuation.
Build a controllable upstream
Instead of disconnecting a real server at an arbitrary moment, create a local fake upstream with a short predetermined response. It can stop before headers, immediately after headers or after a selected content event. This separates error handling defects from network instability and variable generation duration. Save source bytes, the interruption point and expected classification for each scenario. The fake should neither call a paid model nor copy real user content. Its purpose is reproducing a transport sequence, so a synthetic payload is more useful than nondeterministic generation that cannot reliably be interrupted at the same point again.
Distinguish three boundaries
Before headers, the service can still construct an ordinary HTTP error response. Once headers are committed, available signalling mechanisms change; once content is forwarded, the client may already have used a partial result. OpenRouter streaming documentation describes errors in an established stream, although that external format is not automatically an upstream requirement for every provider. Keep receipt of headers, first content and a terminal signal as separate test observations. HTTP status alone does not establish generation completeness: a successfully opened connection can end with a damaged or unfinished stream. These observations must not collapse into a single transport success flag.
Check framing and terminal state
In one case, the fake closes without a final event; in another, it sends a valid error message and completes transport correctly. HTTP/1.1 message completeness depends on the framing described by RFC 9112. A complete HTTP message still does not guarantee completion of the application generation. Test both layers independently. Report the last fully parsed SSE event and bytes already forwarded to the client. If an expected error becomes an endless wait, investigate the timeout and reader state machine rather than simply extending the test until the symptom disappears.
Bound retries and examine the proxy
A proxy can buffer data, change the observed interruption point or make its own upstream retries. The nginx proxy documentation helps check relevant settings in a particular installation. Test the direct path first, then insert the proxy and compare events. All attempts share a maximum budget and receive separate identifiers. If output has already escaped, automatic replay can duplicate content or mix two generations. The test must detect that result even when the second upstream responds successfully. Explicitly test fallback before visible content as well; do not assume safety from an HTTP status alone.
Preserve the reproducible scenario
The artifact includes the fake version, byte fixture, failure offset, proxy configuration, actual attempts and final client status. See streaming errors and inference routing fallback. Bound the suite by duration and scenario count, and confine intentional corruption to the test environment. Passing confirms handling of selected interruption points, rather than resilience to every network failure or correctness of a remote engine. Preserve a minimal case for unexplained differences and separately check whether the test harness changed event ordering. That is more valuable than a large report counting arbitrary restarts.