In this article
A golden fixture captures protocol properties and explicit exceptions instead of requiring identical stochastic prose.
Define the golden reference
A golden inference API fixture consists of an input request, an environment description and expected response properties. Saved generated prose can illustrate a case, but is a poor mandatory equality target: a model update can change wording without breaking the protocol. The reference should answer a specific question, such as whether a required field exists, a stream completes or an identifier remains consistent across frames. A small understandable corpus is easier to inspect than a large collection of accidentally recorded production responses. Each case should explain why it exists and which defect it can detect in the next release.
Separate assertion layers
The first layer checks the HTTP envelope, the second checks JSON schema, and the third checks event sequence and terminal state. OpenAPI 3.1.1 can pin an interface description, but that description alone cannot capture every temporal streaming invariant. For example, a missing required property differs from a null value, as the JSON Schema object reference explains. Streaming also needs checks that content does not appear after completion. Evaluate answer quality separately. Otherwise an arithmetic mistake by the model may block a proxy release even though the transport contract being examined remains correct.
Assemble a small corpus
An initial suite might cover an ordinary response, output limit termination, a tool call, structured output and a controlled failure before response headers. These are selection examples, rather than required capabilities for every engine. Each fixture should contain synthetic messages without personal information or secrets. The OpenRouter API overview provides public response schemas, but an external schema is not automatically a complete upstream onboarding contract. Remove random identifiers from recorded responses using an explicit rule. Erasing every unknown field is dangerous: the normalizer could conceal exactly the change the test was intended to expose.
Update references through a diff
The corpus artifact contains a manifest, schema version, source request hash and expected assertions. When the engine changes, the test first displays a diff: an added field, a new finish_reason or a different accounting frame order. A reviewer accepts the change only after explaining its meaning and compatibility. A command that rewrites every reference must not automatically redefine a new defect as normal. Optional extensions can have a separate allowance policy. Even when an extension is permitted, the report should retain its appearance, so an unnoticed response change does not disappear behind an overall green result.
Apply a bounded release gate
Run the corpus with maximum duration, attempt count and output size. Record the client, engine and tokenizer versions, together with assertions actually executed. An unsupported feature receives an explicit status instead of being skipped as successful. See the benchmark method and stack reproducibility guide. Passing fixtures confirms the enumerated properties for this corpus and environment. It does not establish compatibility with every client, absence of races under load or preservation of model quality. Those limits should appear beside the result rather than being hidden in a release footnote.