AGENTS AND SOFTWARE

From API tools to a browser: what changed on DevDay

Event source: Computer use reaches the Agents API ↗

In this analysis
THE RETROSPECTIVE TAKE

A browser expands reachable software interactions while adding an explicit observation-to-action contract.

Adding a browser changes the kind of state an agent must reason about. A tool can name a record directly; a visual interaction may first need to discover which page and element represent it. The Agents API computer-use expansion brought that difference inside a managed execution product.

The browser arrived after the harness

Codex’s May 16, 2025 launch established earlier cloud tasks. The September 10, 2026 Agents API beta packaged the managed harness. The September 29, 2026 DevDay recap then added computer use. The sequence matters because the new interaction surface expanded an existing execution system rather than replacing every tool with a browser.

Record identity and interface position differ

Consider an application exposing a direct record lookup and a web page listing the same records. A lookup can carry an explicit identifier. The browser path may need to search, inspect a row, open it and verify that the intended record is displayed.

Both paths can fulfill the same business goal while exposing different failure boundaries. A direct lookup can use the wrong identifier; a browser can select the wrong visible object. Treating them as interchangeable calls conceals the observations required to establish identity.

Visual state adds dependencies between actions

A click depends on the current interface, including whether navigation finished or a modal appeared. An earlier observation does not guarantee that a later action still targets the same object.

This does not make browser work inherently unreliable. It means its acceptance contract includes state transitions that a direct tool may represent differently. Preserve the intended outcome and the evidence needed to recognize each transition rather than comparing only the number of invocations.

A paired integration fixture

Use an illustrative task that opens a particular customer record and reads an allowed field. Execute a direct-tool path and a browser path against the same source fixture. Keep the target identity and final answer requirement unchanged.

For the browser case, include a near-identical neighboring record and a delayed page transition. Inspect how the workflow confirms identity before accepting the field. For the direct path, include an invalid identifier. These tests expose different mechanisms without declaring either interface a universal winner.

What the expansion justified

The release justified considering a browser where useful software interactions lacked a suitable direct integration. It did not justify routing every structured operation through visual controls or assuming that managed execution removes interface-specific checks.

The retrospective lesson is to preserve the task while acknowledging different interaction contracts. A browser broadens what an agent can reach, while the application still needs evidence that the reached state is the one the task required.

← All retrospectives