The History Ledger: How to Evaluate AI Agents That Adapt
A benchmark score answers one of three different questions. The history ledger keeps reset, path-dependent and search-selected evaluation claims honest.
Yash Sharma5 min readEvaluation and assurance
Research area 75: Path-dependent, adaptive and contamination-resistant evaluation engineering
In brief
- A benchmark score only answers a reset, history or search-selected question, never all three at once.
- The history ledger declares the question, logs every exposure, then confirms with an independent run.
- Skipping confirmation lets arithmetic alone pick roughly three of four leaderboard winners.
An AI agent's evaluation score answers only one of three questions: how it performs from a clean reset, how it performs after the history it carries into deployment, or how it looks once a search process has already picked it as the winner. Path-dependent, adaptive and contamination-resistant evaluation engineering keeps those three questions separate and answers the one an executive actually needs, not the one a leaderboard reported.
What it is
Evaluation, in this frame, measures a versioned procedure acting on a declared distribution of histories, not a single frozen checkpoint. The procedure can include the model weights, memory, prompts and tools, cache and routing, the evaluator itself and the deployment policy that decides when it refuses, falls back or escalates to a person. For a system that keeps learning after launch, the thing under test is the update rule, not one snapshot of output. For a system chosen by searching over many model and harness combinations, the search procedure itself is under test.
Three principles carry the rest of this essay. Reset accuracy, continuous-history performance and repeated candidate search answer different questions, and blending them into one number destroys the answer to all three. A holdout set stops being protected the moment its labels, its aggregate score or a rollback decision influences a later candidate, indirect exposure counts as much as a leaked item. A replayed trace is a bound on a policy, not the policy itself, and needs a limited live check first.
Agents now act inside their own evaluation, not just answer questions inside it. A controlled study of lifelong agents published in August 2026, PATH-Bench, found task ordering changes measured performance and can flip which agent ranks first, holding backbone and task set constant. The same window produced a study of deep research agents showing that keeping an answer out of the prompt does not keep it out of reach: agents doing live web search retrieved answer-bearing content for withheld questions anyway. By September, a study of self-evolving harnesses found transfer from a development cohort to a withheld one varied with the executor running each stage.
The architecture
I call the operating model that keeps these questions separate the history ledger. It has three parts, and a claim only counts once all three hold.
Declare comes first: name the estimand under test, reset, continuous-history or search-selected, state the distribution of histories the procedure will face, and fix the independent unit, a user, a project or a deployment cohort, before a run starts. Log comes second: every event where a label, an aggregate score or a rollback decision reaches the candidate or its developers gets an entry. The moment a supposedly held-out set appears in this log, it is spent and has to be renamed development. Confirm comes third: only a candidate with a clean log gets a frozen, independent run at the cluster level, ending in one of three honest states, promote with a stated scope, reject or stay inconclusive.
Each part fails differently. Declare fails when a fixed-probability sequential method built for stable odds gets applied to an agent whose success probabilities move as it learns. Log fails exactly as the contamination study showed: a hidden prompt is not a hidden answer when live search can retrieve the same content elsewhere. Confirm fails when a best-of-run or mean-of-several search score gets promoted directly, since two equally capable candidates will show one apparent winner most of the time purely from picking the larger result. A harness that evolved with the weights carries the same risk twice and belongs in confirmation as its own candidate.
Ownership sits with a confirmation function structurally separate from whichever team develops the candidate, holding its own budget, frozen tasks and authority to say no, the same reason a trading desk does not mark its own positions. Fund dedicated deployment-representative histories, ledger tooling that logs every exposure event, and confirmation compute reserved before development starts. Stop promoting from a best-so-far leaderboard number without an independent confirmation run, and stop reusing a set that has already gated rollback decisions while still calling it held-out. Two decisions belong to an executive alone: predeclaring the risk margin and stopping rule before a confirmation run begins, and accepting an inconclusive result as legitimate rather than a reason to keep testing until a number looks better.
What it is worth
Money shows up first in what a search-winning score actually buys. Two equally capable candidates, each tested once, will show one clear winner about three times in four from picking the larger of two coin flips alone, arithmetic, not capability. A program that promotes directly off a leaderboard's best-so-far number is paying for that arithmetic along with whatever real gain exists underneath it.
Confirmation itself is a cost most budgets underprice. Zero observed failures across 30 independent trials still leaves a one-sided 95 percent upper bound on the true failure rate near 9.5 percent, a standard statistical fact, and pushing that bound to 1 percent takes on the order of 300 trials, not 30.
What the evidence actually supports is path-dependence, live-search contamination and harness-transfer variability as real, bounded findings, each tied to one study, not a universal sample size or a portable release threshold. What it leaves open is an affordable way to gather independent deployment histories when they are rare, expensive and constantly evolving, so no return figure is claimed here.
Questions leaders ask
- Why can the same AI agent score well in testing and still disappoint in production?
- Because a reset benchmark and production performance are different questions, not the same question asked twice. A 2026 study of lifelong agents found task ordering changes measured performance and can flip which agent ranks first, even with the same backbone and task set. A clean reset score says nothing about how the agent behaves after the history it actually carries into deployment.
- Does keeping benchmark answers out of an AI agent's prompt stop it from finding them?
- No. A 2026 study of deep research agents found agents doing live web search retrieved answer-bearing content for withheld public benchmark questions, even though the answers were never in the prompt. Protection depends on auditing what the agent retrieves, not only on what it was told.
- Why might a leaderboard-winning AI system underperform once it reaches customers?
- Part of a search winner's advantage can be an artifact of how many candidates were tried, not of the winner's true quality. Even two equally capable candidates, each measured once, will show one apparent winner roughly three times in four purely from picking the larger result. A won leaderboard round needs an independent confirmation run before it earns a promotion decision.
- Is it safe to let an AI agent redesign its own harness or tools in production?
- Only with development and confirmation kept apart. A 2026 study of self-evolving agent harnesses separated a development cohort from a withheld one and found transfer varied with the executor running each stage. Treat a harness the agent helped design as any other candidate: confirm it on unexposed tasks before it ships.
- What is the single most important habit for keeping an AI evaluation honest over time?
- Log every exposure, not just every score. The moment a holdout's label, aggregate result or even a rollback decision reaches a candidate or its developers, that set is spent and has to be renamed development. Most contamination and selection bias enter through exactly this side door, quietly.
