In this article
Validate HTTP framing independently from application termination. Valid JSON in the last buffer does not establish response completeness.
A streaming inference API nests several formats. HTTP carries a body, SSE frames events, and application code interprets JSON inside data. A failure at one layer can resemble a problem elsewhere. A valid object may occupy the final received buffer while transport reports an incomplete message. Validate integrity separately so this cannot become success or an unsupported accusation against the model.
Establish HTTP framing rules
RFC 9112 defines HTTP/1.1 framing, including Content-Length, transfer coding and incomplete-message handling. These rules concern transport messages rather than token or SSE-event counts. Body length is measured in bytes, not characters in decoded text.
Do not invent Content-Length for a stream whose size is unknown. Nor should HTTP/1.1 rules be transferred unchanged to HTTP/2: a route can use different protocol versions on different segments. Name the segment under test because a client’s connection version does not automatically describe upstream behavior.
Preserve header semantics
RFC 9110 defines general HTTP field and content semantics. Successful status is not enough; message consistency matters. When a library reports a framing failure, save its error class and safe metadata instead of replacing everything with “could not parse model output.”
A test fixture should control emitted bytes and connection closure. Use a local fake upstream rather than damage a production route deliberately. Bound malformed variants by known size and duration so missing data cannot cause an indefinite wait.
Create three control cases
The first sends a complete correctly framed body. The second closes before its declared length. The third completes HTTP correctly but omits the expected application outcome. Their classifications must differ: transport integrity and generation completeness answer different questions.
Save original headers, expected size and actual received volume. If a library hides wire details, state that observation boundary. A visible truncated answer does not establish a precise internal server mechanism.
Add SSE above valid transport
The SSE standard defines event extraction from received text. A complete HTTP message can contain an unfinished SSE event if the sender violates application framing. Conversely, some events may arrive completely before a later transport failure.
A result record can distinguish transport_complete, event_complete and application_complete where your implementation defines them. These are proposed artifact fields rather than mandatory external API fields. They prevent partially useful text from becoming a fully completed operation.
Do not retry blindly
Incomplete delivery does not prove upstream performed no work. Automatic repetition must account for POST semantics and ambiguous outcomes. This fixture only needs correct classification and identifiers supporting later investigation.
Related guides cover stream errors and vLLM network reachability. They do not replace framing validation: an open port and successful status alone cannot establish response integrity for the chosen protocol. Retain the exact client version and malformed fixture with the report, since libraries can differ in how clearly they surface an underlying truncation.