MODELS AND MODALITIES

What cached context changed in the Sol and Luna decision

Event source: GPT-6 Sol and Luna broaden the GPT-6 family ↗

In this analysis
THE RETROSPECTIVE TAKE

Caching affects eligible repeated input, so its economic value depends on the serialized workload rather than semantic similarity alone.

A conversation with repeated instructions is not automatically a workload with repeated billable computation. The Sol and Luna launch put caching beside model choice, making the shape of shared context part of the economic decision rather than an invisible implementation detail.

The relevant sequence

The September 22, 2025 Responses article described persistent reasoning across tool work. The API changelog records Astra on September 3, 2026 and Sol and Luna on September 22, 2026. The family announcement also discusses caching for agents and long conversations. That history motivates separating model capability from whether a particular workload reuses eligible input.

Shared meaning is not necessarily shared representation

Imagine an assistant processing requests against a stable policy manual. Each request adds a short customer question, but the application reconstructs the manual with changing timestamps and reordered tool descriptions. People see the same underlying policy while the model receives a changing serialized input.

The diagnostic object is therefore the effective context, not the similarity of the user-facing task. Before estimating a caching benefit, identify which content remains stable, where variable fields appear and what the provider actually reports as cached input. Semantic repetition alone is insufficient evidence.

Compare workload shapes rather than slogans

A policy assistant and a creative drafting tool can use the same model while having different cost structures. The assistant may reuse substantial instructions; the drafting tool may generate long new outputs from brief prompts. A cached-input discount cannot reduce an output component that it never applies to.

Likewise, a stable prefix does not make every downstream operation reusable. Tool results can change, and new evidence can require another continuation. Preserve those boundaries when describing savings. An economic explanation should identify the component affected instead of transferring one discount to the entire task.

A useful accounting artifact

Keep a request ledger separating stable instructions, variable evidence, reported cache usage and generated output. Attach the accepted task outcome so a cheap incomplete response does not look like a successful efficiency gain. Compare equivalent work before and after a deliberate serialization change.

Inspect a miss on an otherwise repeated task. If the effective input changed, investigate construction before changing models. If it did not, consult the documented caching conditions rather than assuming all repetition qualifies. This sequence produces a falsifiable explanation instead of a desired hit-rate story.

What the release changed

Sol and Luna expanded operating choices, while caching made request composition another decision variable. The useful conclusion is conditional: a workload can benefit only through the portions that actually satisfy the caching contract.

Treat repeated context as an observable property of requests. Doing so preserves a connection between the model portfolio, application serialization and the completed work that the organization ultimately pays for.

← All retrospectives