# SSE heartbeats and idle timeouts: what keeps a connection alive

Relate SSE comments, heartbeats and idle timeouts while measuring useful-output delays separately and testing the actual reverse-proxy configuration.

Author: Retrivium team
Updated: 2026-11-01
Canonical URL: https://retrivium.ai/journal/provider-sse-heartbeat-timeout-budget/
Language: en

A model may produce no useful content for some time before its first answer fragment. Proxies between client and engine have their own waiting policies. An SSE comment can keep a connection active, but proves neither answer readiness nor compliance with a request deadline. Measure transport activity and application progress separately, or a long stuck request may appear healthy.

## Define heartbeat meaning

[OpenRouter documents comments](https://openrouter.ai/docs/api_reference/streaming) periodically sent in SSE to prevent connection timeouts. They are format-level service data rather than model JSON. A parser must not pass a comment line directly to JSON.parse or append it to user-visible output.

The [SSE standard](https://html.spec.whatwg.org/multipage/server-sent-events.html) defines comments independently of any platform. Correct handling benefits compatible parsers, while emission frequency and application policy belong to the selected service. One external API example does not establish a mandatory upstream heartbeat interval.

## Separate timers

Measure time to headers, time to first content and gaps between transport reads separately. An application deadline can expire while comments keep arriving. Conversely, computation may be acceptable to the product while an intermediate idle timeout closes the connection sooner.

Record each timer's start and reset event. If comments update transport activity, they must not automatically update last useful output. This matters for observability: an interface can indicate processing while honestly showing a long wait for results.

## Test the actual proxy

The [NGINX reference](https://nginx.org/en/docs/http/ngx_http_proxy_module.html) documents proxy_read_timeout for reading upstream responses. Inspect the actual configuration and directive scope. A value copied from another project does not establish the limit of an entire chain containing other load balancers and client restrictions.

Begin with a fake upstream that sends headers, then comments, while delaying content for a controlled interval. It tests timers without GPU work. Bound all intervals and total duration so the experiment always finishes.

## Compare two traces

One trace remains silent after headers; another sends only heartbeats. Record disconnect time or total-deadline expiry for each. Then add useful content and normal completion. This reveals which behavior belongs to idle policy and which belongs to the overall request budget.

Do not conclude from one successful pass. Record actual arrival gaps, buffering state and observation point. Batched comments may indicate an intermediate-layer issue rather than a fault in heartbeat generation.

## Do not substitute activity for progress

A live connection does not guarantee successful generation. Heartbeats must not increase output-token or completed-response counters. Define excessive-wait stopping policy separately; clients should not wait indefinitely just because service bytes continue arriving.

The [streaming-client guide](/journal/openrouter-streaming-client/) covers incremental reads, while [network and availability](/journal/network-and-uptime/) supplies operating context. The artifact is a timer map for one chain, without a universal SLA or a promise that comments prevent every disconnection. Preserve the tested settings and trace timestamps together so a later proxy update can be compared against the same controlled scenario.