# The conditions behind Opus 5.5 cost-per-task comparisons

How to interpret Opus 5.5 cost-per-task comparisons through effort settings, harnesses and acceptance rules before testing another workload.

Author: Retrivium editorial team
Language: en
Analysis published: 2026-11-03
Event date: 2026-09-22
Original announcement: https://www.anthropic.com/claude-opus-5-5
Canonical URL: https://retrivium.ai/retrospectives/opus55-benchmark-cost-conditions/

## A comparison arrived with conditions

[Opus 5's July 24, 2026 release record](https://platform.claude.com/docs/en/models/opus-5/overview) and [the September 1 Fable and Mythos announcement](https://www.anthropic.com/claude-fable-and-mythos-5-1) provide the immediate family context. The [September 22 Opus 5.5 launch](https://www.anthropic.com/claude-opus-5-5) included benchmark and cost comparisons with stated evaluation conditions. The existence of those conditions is central to interpretation. A published cost-per-task result concerns a defined task, harness and operating setting. It does not establish the cost of an arbitrary repository migration merely because both activities involve coding.

## Effort changes the evaluated system

A model can be evaluated with different reasoning effort settings. A comparison at default effort asks a different question from a comparison of each model's strongest available result. Neither is automatically invalid, but they support different decisions. A deployment team choosing a practical default needs the former kind of evidence; a capability researcher may want the latter. Mixing the two without retaining their settings can make a price-performance claim appear more general than the experiment that produced it. The setting belongs beside the score rather than in a forgotten note.

## The harness carries part of the result

Tool availability, execution limits, available context and retry policy can change what an agent accomplishes. A result achieved with a particular harness therefore cannot be attributed to the model name alone. This does not mean harness effects invalidate every comparison. It means that transferring the result requires understanding which relevant conditions are shared. A team using a different tool set should treat the published result as motivation for a local experiment, not as a measured outcome for its own environment.

## A comparison card preserves meaning

An illustrative evaluation card records the task definition, success rule, effort setting, harness, allowed attempts and observed resource usage. It also identifies whether the number describes successful attempts only or all attempts contributing to the outcome. When a candidate uses an unsupported operation or leaves work incomplete, that observation remains visible alongside its cost. This proposed artifact contains no invented benchmark figures. Its purpose is to prevent a later summary from separating an attractive headline from the conditions needed to understand what was actually compared.

## Transfer the question before transferring the number

The release's most useful effect may be to encourage better cost-quality experiments, rather than provide a universal multiplier for budgets. A local evaluation can preserve the same question while adapting the task to the organization's real acceptance criteria. If its outcome differs from the announcement, that need not imply deception or failure; the evaluated systems may differ. The responsible retrospective keeps the published claim in its original setting and derives a testable local question from it, leaving the local answer to evidence that has actually been collected.

## Related reading

- [Designing an equivalent comparison](/journal/inference-benchmark-method/)
- [Interpreting evaluation uncertainty](/journal/gpu-benchmark-confidence-interval/)