In this analysis
An evaluation baseline needs the model identity, task cohort and acceptance rule that actually produced its result.
A default is a moving reference point
The September 1, 2026 Fable and Mythos launch was followed by Opus 5.5 on September 22. The Sonnet 5.5 announcement on September 28 added another candidate for routine work. A new candidate can influence product defaults, but an interface default and an accepted application deployment are different things. Historical evaluation becomes unreliable when “the default model” is recorded without an identity. The same label may describe different systems at different times, even if the user’s visible workflow appears unchanged.
Preserve what was actually evaluated
An illustrative evaluation compares document extraction across a fixed set of records. If its report records only “Sonnet,” a later reader cannot tell which generation or setting produced the result. Retaining the model identifier, applicable configuration and observation date makes the result interpretable. This is a recordkeeping recommendation, not a claim that a particular alias changed automatically at launch. The important point is that a convenient user-facing name does not provide enough information to reproduce or compare a historical experiment.
A changed cohort can masquerade as improvement
Even with the model identity preserved, the evaluated task set can drift. Removing difficult records or adding mostly easy ones changes the aggregate outcome independently of the model. A team revisiting its baseline should therefore retain the original cohort and identify any new cohort separately. Otherwise a better average may reflect easier inputs rather than better behavior. The same concern applies to changed acceptance rules: relaxing the definition of correctness can produce an apparent improvement without a corresponding gain in useful work.
An illustrative baseline ledger
A practical ledger pairs each evaluated artifact with its model identity, task version and acceptance decision. A candidate upgrade runs against the retained tasks before additional exploratory examples are summarized. If a source file or rubric changes, the record marks that comparison as a different experiment. No score or production transition is asserted here. This fixture creates an explanation for a result that can survive a later interface update and helps distinguish a regression in behavior from a change in what was tested.
Adoption deserves its own decision
A new release can be relevant immediately without becoming the deployment default immediately. Testing, choosing and rolling out are separate events with different evidence. The retrospective lesson is to retain that sequence rather than rewrite old results using the newest product name. A stable baseline is not resistance to improvement; it is the reference that makes improvement observable. Once the candidate is accepted under the intended workload, a deployment change can be recorded explicitly, leaving previous evaluations accurate as historical evidence.