Document-Mediated Policy Adaptation: The Bounded-Edit Ledger

Language- and document-mediated policy adaptation is the practice of changing what an AI system does by editing the document that governs it, a skill, a prompt, a procedure, rather than retraining the model underneath. Three 2026 studies show a frozen executor can be steered a long way by a versioned document, but none show an edited document can be trusted without a trial against its own parent. An executive who skips that trial buys a metric, not a capability.

What it is

A skill, here, is a document written to be executed by interpretation rather than by a compiler: it states when it applies, what steps to take, which tools it may call, and what a correct outcome looks like.

The area separates six things easy to blur into one word, learning: retrieving a memory, extracting a rule from one experience, selecting among existing skills, editing one skill within a bounded scope, training an editor model to propose edits, and retraining the executor's own weights. A team that says an agent learned from experience, without naming which of these six happened, has said nothing actionable.

Four principles hold the area together: the executor's weights stay fixed while the document changes, edits are structured and bounded against a specific base version rather than free rewrites, an edit is worth nothing until validated against its own parent on matched tasks, and permissions must live outside the editable text so a persuasive paragraph can never grant itself capability it was not already given.

Three 2026 papers anchor the evidence. DMRL (September) described an online advertising system that freezes its executor and trains a separate editor and outcome predictor, reporting lifetime value up 0.052 percent alongside a secondary metric that fell 0.020 percent. SKILLER (August) described an actor and critic process editing reusable skills for a frozen executor. SkillOpt, a May Microsoft paper, described bounded add, delete and replace edits accepted only when they clear a validation bar. All three are preliminary author claims by their own reviewer, DMRL's numbers first party with no extracted sample size or uncertainty interval, and none shows an edited document transferring to a different model or tool version, or a durable, general return: worth a pilot, too immature to fund as finished.

The architecture

The operating model I use is the bounded-edit ledger: every change to a governing document is proposed, checked, trialed against its own parent, and either promoted or archived, with full history kept for rollback.

The parent document is the current, trusted version of the skill, executed by a task model whose weights stay fixed throughout. An editor, a second model, a critic process or a human reviewer, proposes a bounded edit against that parent, scoped to a limited span and tied to a specific base version. A separate authorization check confirms the edit stays inside permissions the system already grants, since permission never lives inside the document.

The edit is applied to a copy, a candidate that sits alongside its parent. Candidate and parent run a paired trial on matched tasks, isolating the edit from the day's traffic. If the candidate clears a predetermined margin with no regression in any mandatory safety check, it is promoted and becomes the new parent. If not, the parent stays in force and the candidate is archived, not discarded, with its diff and validation report.

Two failure modes recur most: promoting on one favorable run instead of the paired trial, the same overfitting as repeatedly tuning against one validation set, and letting the document's own language carry authorization it was never granted.

How to lead it

Split ownership the way you would split who drafts a contract from who signs it: the team that writes the editor should not control the trial that judges its edits.

Fund four things: the editor or critic, the paired trial for every candidate, an archive of every rejected edit with its diff and report, and a compatibility check that reruns procedures whenever an underlying model or tool version changes, since a cached validation result is tied to the version it was tested against. Skipping any of the four moves its cost downstream to an incident.

Two decisions belong to the executive alone: setting the promotion bar, the smallest improvement worth accepting and largest regression tolerated, before any trial result is visible, and deciding what stays permanently outside the editable document, the permissions, spending limits and irreversible actions no edit may ever grant itself. For a stable, low-volume task, the right call may be a single, well-reviewed static document instead.

What it is worth

The money shows up in four places: editor or critic calls spent proposing changes, trial traffic spent validating them, the token cost every edited document adds at every future execution, and the cost of a promoted edit later rolled back. None disappear by calling the mechanism zero extra training: a document that grows two hundred words to cover an edge case charges that cost on every call afterward.

The evidence supports funding a governed pilot where reuse is high and tasks validate cleanly, and supports the claim this can move a measured metric, since DMRL reported exactly that in one deployment. It does not support a general return figure, a claim that edits transfer across models or tool versions, or that a rising validation score alone proves lasting value. Measure with an audit set the editor never sees.