# Slow SSE clients: backpressure and bounded memory

Test inference streams with slow readers through backpressure, bounded queues and memory, transport flow control and independently verified generation cancellation.

Author: Retrivium team
Updated: 2026-11-01
Canonical URL: https://retrivium.ai/journal/provider-sse-slow-reader-backpressure/
Language: en

Fast consumers hide streaming-server problems: output leaves queues immediately and memory appears stable. A slow client creates another mode. Generation may continue while the application accumulates unread fragments. Providers need a bounded slow-reader test observing queues, memory and operation completion. It checks the transport path rather than peak model performance.

## Locate accumulation points

[MDN describes backpressure](https://developer.mozilla.org/en-US/docs/Web/API/Streams_API/Concepts) through stream and queue behavior. An inference chain can buffer at the engine, API process, proxy and client reader. Bounding one layer does not establish bounds elsewhere. Map where data originates, waits for delivery and can be stopped.

Network pressure is not automatically generation cancellation. An API that drains upstream quickly into its own queue may prevent transport pressure from reaching the engine. That architecture needs separate memory limits and policies for slow connections.

## Separate transport from application

[RFC 9113](https://www.rfc-editor.org/rfc/rfc9113.html) defines HTTP/2 flow control. It governs data transfer within its scope rather than model-task queues or application policy when a buffer fills. HTTP/2 support alone cannot establish safe inference-service capacity.

Define observations at each layer. Application measures include in-flight requests, queue size and delivery waiting. Clients record read intervals and total bytes. Values describe the selected experiment rather than universal limits for other models.

## Build a bounded slow reader

First run a short control trace without delays. Then read the same synthetic stream with predetermined pauses. Bound event count, total size and overall duration. A GPU-free fake upstream is useful for initial transport investigation.

Preserve memory and queue time series rather than only start and end values. A short peak and continuous growth need different responses. Stop at the defined bound and record that limit rather than continue until the machine fails.

## Check the proxy's role

The [NGINX reference](https://nginx.org/en/docs/http/ngx_http_proxy_module.html) describes buffering and response forwarding. Enabled buffering can change where data accumulates; disabling it does not remove every other queue. Record actual settings and compare direct access with the proxy route separately.

If client delays cause growth on only one path, localize that segment. Do not disable protective limits merely to make the test pass. The goal is explainable bounded behavior rather than indefinitely supporting a consumer that stopped reading.

## Confirm cleanup after stopping

After client closure, verify the corresponding operation ends at observable boundaries. A lower connection count does not establish that upstream generation stopped. Queue and in-flight measures should return to the expected state, or the report must identify missing evidence.

Related guides cover [continuous batching](/journal/continuous-batching/) and [capacity planning](/journal/inference-capacity-planning/). They concern model load; slow-reader testing examines another dependency and prevents confusing spare compute with safe transport handling. Keep pause settings, buffer limits and the exact proxy path alongside results so an update can repeat the same experiment without changing its demand pattern.