# Testing SSE buffering at a reverse proxy

Measure reverse-proxy SSE buffering through direct and proxied paths, proxy_buffering, X-Accel-Buffering and an event-arrival timeline.

Author: Retrivium team
Updated: 2026-11-01
Canonical URL: https://retrivium.ai/journal/provider-proxy-sse-buffering-test/
Language: en

A model may generate incrementally while clients receive large batches. This does not prove the inference engine ignores streaming. Buffers, transport layers and client processing sit between source and reader. Compare the same controlled stream directly and through the reverse proxy, preserving send and receive times. One visual observation cannot identify the cause.

## Start with a known source

Use a small fake upstream emitting several SSE events with predetermined pauses. Known content and timing remove model queues, GPU behavior and variable output lengths from the experiment. Bound total duration and bytes beforehand.

The [SSE standard](https://html.spec.whatwg.org/multipage/server-sent-events.html) defines event framing rather than packet-delivery frequency. Measure complete-event arrival at the client instead of requiring read sizes to match sent lines. Otherwise ordinary transport coalescing becomes a false protocol violation.

## Preserve the direct control trace

Connect to upstream within an authorized test boundary without the proxy under investigation. Record each source send time and client complete-event arrival. Save client version and reading method. Client-library buffering can distort results before NGINX enters the path.

If direct traffic already appears only at the end, investigate reader and source first. Changing proxy settings may alter results incidentally without explaining the original problem. A control trace is evidence rather than a formality.

## Inspect actual NGINX settings

The [proxy-module reference](https://nginx.org/en/docs/http/ngx_http_proxy_module.html) describes proxy_buffering and X-Accel-Buffering where header processing permits it. Effective location behavior can differ through inheritance and other directives. Preserve the route's effective test configuration without secrets.

Do not copy a universal “disable everything” recipe before measuring. Change one material factor and repeat the same trace. If the application sends a header, verify that the proxy honors it; an application response line alone does not establish intermediate-layer behavior.

## Compare delay patterns

The [MDN Streams guide](https://developer.mozilla.org/en-US/docs/Web/API/Streams_API/Concepts) explains variable-sized chunks. Record complete-event time, sequence number and request path. Multiple events in one read are not inherently invalid; a repeatable additional delay relative to the control is the relevant observation.

Compare event gaps and time to first useful content. Do not count service comments as answer tokens. Mark cold connections separately because transport establishment differs from buffering already emitted events.

## Bound the conclusion to the tested path

If a buffering change removes delay on this location, it explains that configuration rather than every future route. Another load balancer, compression layer or client reader can alter delivery again. Retain the fixture with settings for post-update checks.

Related material covers [Runpod HTTP proxying](/journal/runpod-http-port-proxy/) and [vLLM streaming validation](/journal/vllm-openai-streaming-test/). The artifact connects symptoms with one transport path rather than establishing universal generation speed or external-router requirements. Preserve both successful and failed traces so later reviews can identify what changed instead of relying on a screenshot that only shows the final answer.