Skill Engineering: The Three-Gate Contract for AI Agents
Agent skills are mixed evidence, not a proven lift. The three-gate skill contract separates discovery, execution and outcome so leaders fund what works.
Yash Sharma5 min readAgentic AI in production
Research area 20: Skill and reusable-capability engineering
In brief
- Skills only pay off when each one clears three separate gates, not how many a catalog has collected.
- The three-gate skill contract scores discovery, execution and outcome separately so leaders see which gate failed.
- Catalog growth without paired evidence adds token cost and risk, since 2026 studies show gains are inconsistent, not universal.
Skill and reusable-capability engineering packages what an AI agent knows how to do into a versioned artifact it can load, run and be judged on, separate from the model underneath it. What decides whether this pays off is not how many skills it has collected, but whether each one clears three gates: found correctly, executed correctly, and shown to improve the task's actual outcome, not a score its author wrote. Leaders who fund catalog growth as a proxy for capability are optimizing a number the 2026 evidence does not support.
What it is
A skill is an external artifact containing procedures and optional executable resources that an agent loads at the moment it needs them. That excludes a tool endpoint the agent calls directly, knowledge baked into the model's weights, and an agent's accumulated memory. A skill can call or describe a tool, but the two are not the same, and treating any packaged file as a tested capability is the first mistake here.
A skill needs identity, scope and evaluation evidence before it earns a catalog place, not a plausible description. Permission to act, the authority object, comes from the host system, never from the package's own text, and every package stays untrusted until reviewed and run inside a boundary it cannot loosen.
Catalogs are growing faster than the evidence that any given skill helps: an NVIDIA evaluation of enterprise production skills scored 947 paired cases across 58 of 64 skills and four harnesses, reporting a composite lift of about 0.21, graded C, one vendor's own measurement. A separate study matched 117 skill-project pairs across four models and found adding a skill lowered the mean pass rate on functional tests while token use rose 72 to 394 percent, graded B, with matched controls. The two do not average into one figure, but together reject any claim of a universal lift.
The architecture
The clearest way to run this is what I call the three-gate skill contract: every skill must clear discovery, execution and outcome as three independently measured gates, because a single composite score hides which one failed.
Gate one is discovery: it must filter by permission before it ranks by relevance. Two similarly described skills, one internal, one external, can score as identical matches. Only a permission filter, checked before ranking, tells them apart, and an unspecified destination should trigger a request for clarification, not a guess.
Gate two is execution, run against a pinned version of the skill inside a workspace scoped to just that task's inputs and permissions. It answers whether the procedure ran as written, not whether the task succeeded. Selected correctly seven times in ten, succeeding nine times in ten when correctly selected, but still succeeding two times in ten on a wrong selection, the true end to end rate is 69 percent, well under what gate two alone shows. Reporting gate two alone as overall reliability is how this area misleads a leadership team.
Gate three is outcome: an independent check of the task's actual end state, not the skill's own account. A separate study compiled written procedures into typed, executable harnesses for thirty skills, raising measured step adherence from 56 to 86 percent using 0.58 times the tokens: evidence a compiler can enforce a procedure's mechanical parts, not that it encoded the right procedure. The most common failure across the studies is skipping from a relevance match straight to an outcome claim, with no gate two check between.
How to lead it
Ownership belongs with whoever grants the effect a skill is allowed to have, not whoever wrote it first. A loader must reject unmet prerequisites before importing a skill's code, since importing a script can execute it.
Fund the paired evaluation itself, discovery, execution and outcome as three separate scores, an authoring budget honest about amortization since a model upgrade can erase it overnight. Stop funding catalog size as a success metric. Automatic skill creation stays quarantined against a no-skill baseline until proven on unfamiliar task families.
Two decisions cannot be delegated. Where the effect boundary sits for a class of skills, which actions it may take against systems, money or data, is a risk decision, not a modeling one. The promotion margin, how much held out lift a skill must clear and what counts as a critical failure, must be set before results come in, not after.
What it is worth
Token overhead can outweigh the gain, as the web development study found with cost increases of 72 to 394 percent alongside a falling pass rate, so cost must be tracked per verified success, not per token processed. Authoring cost pays back only after many repeated uses, once maintenance and review time are counted honestly. Risk shows up early: a skill's description can dominate selection, and a hidden dependency can act before review.
Measure with paired, length matched comparisons against a no-skill baseline and a fixed workflow, reporting discovery, execution and outcome separately. The 2026 evidence supports that skills help on some task and model combinations and hurt on others, and that compiling procedures raises measured adherence to them. It does not support one transferable lift figure, a proven case that skills beat fine tuning or fixed workflows at equal effort, or a track record for safe, unsupervised catalog growth. No return figure appears in the source evidence, and none is invented here.
Questions leaders ask
- What is a skill, and how is it different from a tool an agent calls directly?
- A skill is a packaged set of procedures and optional resources an agent loads at the moment it needs them, evaluated on whether it was found, executed and produced the right outcome. A tool is a fixed, typed operation the agent calls directly. A skill can describe how to use a tool, but the two are separate ideas and should be evaluated separately, not treated as interchangeable.
- Does adding more skills to an agent make it more capable?
- Not automatically. A 2026 study of web development tasks found that adding a skill lowered the mean pass rate on functional tests across four models while raising token use by 72 to 394 percent. A larger catalog increases search and review cost and is not, by itself, evidence that an agent can do more. Each skill needs its own paired evidence before it earns a place.
- Should a skill be measured by how often the agent selects it?
- No. Selection tells you whether discovery worked, not whether the task's outcome improved. The three-gate skill contract keeps discovery, execution and outcome as separate scores, because a skill can be selected correctly and executed exactly as written while still producing the wrong final result if the procedure itself encoded a mistaken interpretation.
- What is the biggest security risk in a skill catalog?
- A careless or malicious description can dominate which skill gets selected before any code runs, and a skill's transitive dependencies can perform hidden actions the review never saw. Treat every package as untrusted until reviewed, resolve permissions outside the model rather than from the package's own text, and reject unmet prerequisites before importing any of a skill's code.
- How do we decide when a skill is worth keeping in the catalog?
- Keep a skill only when held-out testing shows it clears a lift margin set in advance, measured against a no-skill baseline and a fixed workflow, and when its authorization and dependency review have not turned up a critical violation. Skills that fail either test should be retired rather than kept on the chance they help somewhere unmeasured.
