Why AI Agents Need Separate Goals, Metrics and Permissions

Intent, objective and decision-policy engineering writes down what an AI system is meant to achieve, separately from how it is measured, what it is forbidden to do and who it may ask before acting. Most automation failures that reach a board trace back to one of those blurring: an unwritten goal, a metric that quietly became the goal, or an assumed permission.

What it is

A good specification keeps five things apart. The outcome is the real-world state someone wants, independent of how it will be checked. The proxy is the observable stand-in for that outcome, useful only as a diagnostic. A hard constraint must be satisfied regardless of how good the rest of the answer looks, such as staying inside an authorized account or never disclosing a customer's record. A soft preference is a tradeable priority, and a permission is a separate grant of authority, never implied by a beneficial-sounding goal.

Four principles follow: a proxy is not the outcome, an unknown stays unknown rather than getting an invented answer, a prose-only policy is not enforced until something outside the model checks it, and authority is not earned by intention. A 2026 paper on context engineering frames this area around goals, values and a trade-off hierarchy, a useful lens, though one author's proposal, not a tested ordering.

Agents now act on real accounts, turning a blurred goal into an unsupervised decision with real financial or legal consequences. A coding benchmark of 419 professional tasks, each missing one to three requirements, tested across six models, found models routinely complete confidently on incomplete instructions rather than asking what was left out, using synthetic deletions, not real negotiation. A study across nine AI safety gridworlds found reinforcement learning could widen the gap between reported reward and actual performance. A third study of twenty-one permission interfaces and five commercial products found the interface and its enforcing code are frequently separate software, the gap where revoked permissions keep being acted on.

The architecture

Call it the four-gate decision model: feasibility, clarification, ranking and commit, the order every consequential decision should pass through.

The feasibility gate asks only whether an action satisfies every hard constraint and current authorization, before any value is weighed. If the feasible set is empty, the system should report infeasibility, not quietly return the least-bad option: requiring arrival within three days and a cost under ten units, when the options cost twenty for two-day delivery or five for five-day, leaves nothing that satisfies both, and shipping the five-day option anyway changes the task without telling anyone.

The clarification gate asks a question only when the answer would change the chosen action, priced against the value of the information. In an illustrative calculation, two goals are plausible at sixty and forty percent, the right action worth ten and the wrong one costing twenty, so guessing nets negative two, while a question resolving the uncertainty for one unit of cost is worth eleven net. A slow question can still destroy that value.

The ranking gate chooses among what remains feasible. A scalar utility works when trade-offs are commensurable, a lexicographic order when a higher priority must never be sacrificed for a lower one, a Pareto presentation exposes a trade-off without picking a winner, and a risk-sensitive policy needs a declared loss distribution and tolerance, a business decision, not one the model sets for itself. Its failure mode is proxy gaming, a measurement rising while the tracked outcome stays flat.

The commit gate revalidates authority immediately before the action executes and binds the decision to a specific version of the intent record. When intent changes materially, every downstream plan built on the old version should be invalidated, and an effect already committed may need a compensating action.

Ownership mirrors the model, not one team. A core policy and authorization service owns hard constraints and permissions, the only thing that can issue a grant. A business or product owner owns the outcome and soft preferences. A model only proposes interpretations and rankings inside both, never converting an objective into a permission for itself, the rule worth writing into every design review. Two decisions belong only to an executive: acceptable loss behind any risk-sensitive gate, and a stakeholder conflict the system correctly refused to invent an answer to.

What it is worth

The money, risk and time show up in three places: a wrong action at machine speed and scale, a clarification process intrusive enough that people abandon it, and a prose-only policy that turns out, after an incident, never enforced. None of the 2026 evidence isolates a return figure for a structured intent record, and none should be presented as one.

Measure before and after with a paired comparison at the task or run level, and predeclare the smallest improvement worth having and the largest tolerable regression. One bound worth keeping in mind: zero violations across thirty independent trials is consistent with an underlying risk as high as roughly nine and a half percent, and even zero across three hundred trials still permits roughly one percent.

What the evidence does support is that separating elicitation, policy and enforcement improves auditability, and that ambiguous tasks remain genuinely hard for today's models. What it does not yet support is that a structured intent record causally beats a good prompt under identical enforcement. That comparison has not been run, and the gain stays a hypothesis.