The Runtime Ladder: How Much AI Agent Infrastructure to Build

Harness, runtime and execution-environment engineering is the discipline of building the execution boundary around a language model: the layer that turns a model's proposed action into observed, authorized and verified work. Most production incidents in agentic systems trace back to a runtime that was either too thin to catch a false completion or too elaborate for the task it was actually given. Getting this boundary right is an infrastructure decision with a direct line to cost, risk and how fast an organization can trust an agent with real systems.

What it is

A harness is the execution boundary connecting four parts: what the system observes, the model's proposed next action, the tools that carry it out, and the decision that declares work complete. A model only proposes, and a proposal has no authority until something outside the model validates it.

Four principles hold it together. A model's own notes, tool descriptions and retrieved text must stay outside the boundary that grants authority, since treating a plausible narrative as fact is how false completions enter production. Isolation must be enforced by the execution platform itself, on processes, files, network access, credentials and resource quotas, not by an instruction asking the model to behave. Completion is a checked property, not a status code: a tool call can report success while writing to the wrong place or writing nothing, so completion requires the expected artifact to exist, match its declared identity, pass required checks and reconcile with whatever external system it touched. And the machinery a task needs should match the task, not a default: a bounded lookup does not need the runtime a multi-day workflow needs to survive a crash and a weekend of waiting.

Agents now run for hours, touch real external systems, and keep working after the person who started them has moved on, which is why 2026 engineering reports show teams building this boundary deliberately rather than assuming a capable model makes it unnecessary. OpenAI's February 2026 account of its own harness describes isolated worktrees and mechanically enforced architectural boundaries, an architecture example rather than a measured 2026 speed gain. Anthropic's March 2026 report on harness design describes removing parts of an earlier harness once a newer model no longer needed them, illustrated by an application-building run on Opus 4.5 and 4.6 that cost about $124.70 over roughly three hours and fifty minutes, a single vendor example with no randomized comparison across models or harness designs. Anthropic's April 2026 report on scaling managed agents describes separating the model harness, execution environment and session history to reduce operational coupling, again a first-party account, not independent proof of comparative performance.

The architecture

My model for this decision is the runtime ladder: five rungs of increasing state and enforcement, with one rule attached to all five, that a task climbs only as far as it actually needs.

The first rung is a stateless call: a request validator and an output check, for a bounded extraction or classification with no lasting effects. The second is a fixed workflow: a known sequence of typed transitions for a repeatable process whose steps rarely change. The third is a minimal tool loop: observations, a budget and a gate on every tool effect, for work that is bounded but genuinely uncertain in its path. The fourth is a durable runtime: an external event history and reconciliation so a task can survive a crash, an approval delay or a long wait. The fifth is a managed runtime: a hosted version of the above that still leaves the organization responsible for authorization, verification and correctness, since a provider's shared method name does not guarantee the same behavior underneath.

Below the rung a task needs, effects go untracked: a crash between an action and its acknowledgement leaves the runtime unable to tell whether an external system already recorded it, and the safe response is to treat that gap as unresolved, not to guess. Above the rung it needs, an organization pays for overhead a simple job never touches and builds a harness harder to change later than the model it was built around. At every rung, completion means the expected artifact exists, matches its declared identity, passes the required checks and reconciles with whatever it touched outside the runtime: a successful status is not the same claim.

What it is worth

The money and risk show up in three places: external effects duplicated or lost because a crash was never reconciled, infrastructure cost spent running durable, managed machinery for tasks that only needed a stateless call, and the downstream cost of a false completion that reached a real system before anyone checked it. Measure the reconciliation rate on external effects, the false completion rate before and after a verifier is added, and cost per independently checked outcome, not cost per call or token.

Two decisions belong to executives, not engineers alone: approving removal of a deterministic authority gate, since a model seeming more capable is evidence about skill, not about how much authority it should get, and setting, before a runtime change ships, how much unresolved or duplicated external effect the business will tolerate while reconciliation catches up. When a model changes, remove one harness component at a time and retest, rather than assuming the whole stack can shrink at once.

The 2026 evidence supports using the reported architectures as implementation references and supports removing a harness component and retesting after a model change. It does not support a general claim that a fuller harness produces a productivity gain across models, harder tasks or independent evaluators at a fixed budget: cross-model, fixed-budget evidence here is still insufficient, and no return figure beyond an organization's own reconciled outcomes should be treated as established.