In this article
Request bytes and model tokens are bounded at different layers. One limit does not replace the others or establish post-decompression protection.
A service may reject a request before model execution because its body is too large. Another request passes the HTTP bound but exceeds context after message preparation. Both can look like “input too long,” but their remedies differ. Providers need separate size, parsing and model-budget boundaries so failures are not automatically attributed to GPUs or tokenizers.
Define each limit’s unit
The NGINX core reference documents client_max_body_size and related body-handling settings. It bounds HTTP body bytes at that layer, not model tokens or prepared chat-prompt length.
Name transport-document size, parser limits and model-sequence limits in the interface record. Make units explicit. Characters, bytes, document pages and tokens cannot be exchanged through one fixed multiplier for all languages and inputs.
Check Content-Encoding handling
RFC 9110 defines content-representation semantics, including coding. Actual compressed-request support depends on implementation and configuration. Do not assume every proxy accepts, decompresses and safely bounds such input automatically.
Where compressed bodies are supported, identify the decoding stage and the bound applied afterward. Small wire size does not establish low decompressed memory use. Use finite safe fixtures rather than uncontrolled expansion merely to illustrate the difference.
Separate JSON from the model
RFC 8259 defines JSON grammar, but valid syntax can still violate an API schema. Parsing is followed by checks of messages, roles, types and requested capabilities. Only then can the model input be assessed with its template and control elements.
Record the first rejection stage rather than only the final client message. If body acceptance failed, VRAM does not explain the result. If context is exceeded after validation, increasing the proxy body limit will not fix it and may only permit unnecessary processing.
Construct controlled fixtures
Use a short valid request, a document above the byte bound, and an input within bytes but beyond the declared token budget. Select sizes for the local test environment and record them as experiment settings rather than universal provider recommendations.
At each step save status, error class, observation point and admission to generation where visible. Do not demand identical error wording across layers. The important property is distinguishing transport size, JSON shape and model limits while leaving hidden stages explicitly unknown.
Recheck order after updates
Proxy, parser and chat-template changes can move observed boundaries. Repeat the small set with release checks and save effective settings. When limits conflict, fix policy and explanation rather than only one convenient example.
Related guides cover max-model-len and long context. They address model behavior; HTTP fixtures prevent conflating it with wire-document size or unsupported decompression assumptions. Keep the expected rejection layer with every fixture so a changed status cannot silently reclassify an early transport rejection as a successful model-level validation.