vLLM chat templates: fix string versus OpenAI content formats

Fix a content format mismatch by aligning the client message with the model’s chat template. A template may expect a plain string or a list of typed parts. Confusing those representations can produce the wrong prompt even when the request is valid JSON. Inspect the transformation before changing the server configuration or its GPU environment.

Locate the representation boundary

A simple text message carries a string in content. An OpenAI-style representation can instead contain objects describing individual text or media parts. Iterating over parts and inserting a string are different template operations. The vLLM documentation describes automatic detection and the explicit chat-template-content-format override. This controls message conversion; it does not change model weights, expand context, or establish support for every possible input modality.

Capture the evidence first

Keep a minimal request without images, the tokenizer revision, and the actual template used by the process. Record whether the client sends a string directly or an intermediary library constructs the parts array. Inspect the serialized request rather than relying on the application’s in-memory object. Confirm that a startup template override matches the file being reviewed. A local template can change independently of the checkpoint, leaving an otherwise reproducible deployment with a different message transformation.

Build a diagnostic request pair

Use the instruction “Return the word ready” in two separate requests with the same role. Send one as a string and the other as an array containing one text part. These are synthetic diagnostic inputs, not quality measurements. If only one representation fails, compare its rendered prompt and the format-detection log. If both fail identically, inspect the role sequence and template syntax before concluding that content format is responsible. Keep all sampling settings unchanged during this comparison.

Accept the conversion, not just the response

Repeat the corrected path with system, user, and assistant messages, followed by a conversation containing several turns. The Hugging Face guide treats templating as conversion from messages into a token sequence. Check text preservation, part ordering, and role boundaries. A successful request should not leave literal dictionary representations in the prompt, omit a text fragment, or duplicate control delimiters. HTTP success alone cannot establish that the template interpreted the conversation as intended.

Use an explicit override deliberately

An explicit format is useful when detection is ambiguous or a tested deployment needs fixed behavior. Store that choice alongside the template revision. It does not validate image, audio, or tool messages, which have additional contracts. Keep an input-to-rendered-prompt fixture as the acceptance artifact for the next upgrade. Include empty text and Unicode inputs to exercise conditional branches that a single ordinary message misses. The report should show what reached the model, making future template changes observable before they become an unexplained change in answer quality.

Testing streaming responses from the vLLM API; vLLM served-model-name: Fix a Missing API Model.