# A bounded cancellation soak test for request and connection leaks

A bounded inference completion and cancellation test with resource baselines, recovery pauses, stop conditions and investigation of suspicious growth.

Author: Retrivium team
Updated: 2026-11-01
Canonical URL: https://retrivium.ai/journal/provider-cancel-soak-resource-leaks/
Language: en

## State the hypothesis

Cancelling a streaming request can stop client consumption before the engine releases its work. A defect on that path may leave active requests, connections or buffered data after repeated cancellations. A soak test examines a specific hypothesis about residual state rather than attempting to overload the server indefinitely. Record versions, model, output limits and known background tasks before execution. Establish a quiet baseline after warmup: loading weights and creating initial memory pools are not leaks by themselves. Also identify which resources should return to baseline and which may legitimately remain cached.

## Alternate completion and cancellation

The corpus contains short completed generations and cancellations at defined phases: before first content and after content appears. [AbortController](https://developer.mozilla.org/en-US/docs/Web/API/AbortController) can cancel fetch and response body consumption, but does not guarantee immediate cancellation of remote inference. Record the cancellation phase, confirmed server events and terminal state for each attempt. Bound cycles by total request count, duration and output volume. Do not submit another group after the preceding one has already exceeded the active work threshold. Otherwise a diagnostic test becomes an uncontrolled queue source and makes the original defect harder to explain.

## Measure recovery during pauses

After every request group, provide an equal observation period without new work. Compare in-flight requests, queue length, open connections, process memory and available engine metrics. The [vLLM metrics documentation](https://docs.vllm.ai/en/stable/usage/metrics/) helps select observations, but verify metric names against the installed release. Charts should mark cycle and pause boundaries. Stable elevated memory does not equal a leak: an allocator may retain allocated blocks. A stronger signal is monotonically increasing residual state accompanied by requests or objects that fail to recover after comparable pauses. Even that signal first requires excluding unrelated traffic and expected cache changes.

## Separate client and server

Client memory can grow because of unread chunks or accumulated diagnostic logs rather than a defect inside the engine. [Streams API concepts](https://developer.mozilla.org/en-US/docs/Web/API/Streams_API/Concepts) explain queues and backpressure; the test reader should explicitly finish or cancel consumption. Correlate client descriptors with server records without assuming equal counts in the presence of pooling. [Prometheus practices](https://prometheus.io/docs/practices/instrumentation/) help observe current work separately from completions. The diagnostic branch is straightforward: if the server recovers but the test process keeps growing, inspect the harness. If server work remains active after acknowledged cancellation, investigate cancellation propagation and resource release through the service chain.

## Preserve the limits of the result

The report contains executed cycles, cancellation points, recovery intervals, stop reasons and resource charts with a baseline. See the [GPU monitoring scorecard](/journal/gpu-monitoring-scorecard/) and [inference capacity planning](/journal/inference-capacity-planning/). No growth in a bounded experiment confirms only the examined paths and duration. It does not exclude a rare race, a leak in another endpoint or accumulation under a different context length. Investigate suspicion with another controlled run and profiling rather than extending the test indefinitely. A mandatory experiment limit preserves reproducibility and explains why execution stopped at the selected point.