Streaming tool calls: assembling arguments by index and ID

A streamed tool call arrives as updates rather than a ready function invocation. Its name, identifier and arguments string may not arrive together. A naive handler parses every fragment as standalone JSON or concatenates all arguments into one buffer. Both mistakes become obvious with multiple calls. Provider validation needs a small deterministic assembly procedure separated from tool execution.

Choose an assembly key

The OpenRouter API schema describes tool_calls in responses and corresponding tool messages. During streaming, keep a separate record for each call within its choice. Use the index to associate updates and preserve the ID as the application-level invocation identifier. Check the exact delta shape against the selected interface and version.

Do not assume every event repeats the function name and ID. If an implementation emits them initially, later fragments should extend the same record. A missing ID in the first observed fragment must not be repaired with one shared placeholder for every call.

Separate strings from objects

Arguments can be a string containing JSON at the protocol boundary. A fragment of that string does not have to be a valid document. First assemble the call data in order, then parse the completed value and validate it against the tool schema.

The vLLM tool-calling documentation shows dependencies on model and parser. A plain-text test therefore cannot establish tool suitability. Fixtures must cover the call shapes the service intends to advertise.

Create a two-call example

Suppose a fixture contains calls indexed 0 and 1. Fragments interleave: part of the first argument, the second call’s beginning, continuation of the first, then completion of the second. The expected result is two independent arguments strings and two preserved identifiers. A shared concatenation buffer must fail this fixture.

Functions perform no actions in the test. Names, arguments and call associations are enough to validate. This isolates parser work from external actions, real accounts and user secrets. Bound the event count and accumulated buffer size beforehand.

Test transport boundaries

A network read is not a tool delta. The SSE standard defines its own event framing. Bytes first pass through decoding and framing; only then does application code receive data for call assembly. A closing brace in an arbitrary network buffer is not proof of function completion.

Add cuts inside strings, escaped quotation marks and multibyte characters. The final expected value must remain unchanged. Invalid JSON in genuinely completed arguments is different from simply waiting for the next fragment.

Define the execution boundary

Execution begins only after confirmed completion and argument validation. A correct function name does not authorize every action. Tool authorization remains separate from API conformance testing. A disconnected stream must not turn partially assembled text into an executable command.

The tool-calling guide covers fundamentals, while vLLM streaming tests cover transport. The deliverable here is a reproducible assembly fixture, not a promise that every engine or model supports identical calls.