Separate Structure From Tuning Before You Trust an AI Design
When AI proposes a system, split the fixed design from its numeric tuning and compare candidates only at matched effort. Here is how.
Yash Sharma5 min readModel architecture and training
Research area 53: Nested program-structure and numerical optimization
In brief
- Split an AI-proposed system into fixed structure and tunable parameters before judging it.
- The split-and-solve model routes each parameter to a matched solver and rejoins at a budget-matched verdict.
- Evidence is early, worth a pilot on one decision, not a wholesale replacement of your optimization process.
Nested program-structure and numerical optimization is the discipline of separating what an AI-proposed system is built to do, its structure, from the numbers tuned inside it, its parameters, then matching each numeric decision to whichever method fits it. Executives meet this the moment an AI agent proposes a working program, a pricing engine, a control policy, a trading rule, rather than a plan. Treating the whole proposal as one score to accept or reject hides that structure and tuning fail and cost money in different ways.
What it is
A structural sketch fixes the syntax, control flow, and operations a proposed system may use. Inside it sits a manifest of named numeric parameters, each with a type, bounds, a scale, a default, and units, the holes a solver fills. The line is not absolute: changing something as small as a polynomial's degree can open new holes, so it is a decision a team writes down for a system, not a fixed property.
Four principles follow. Judge two proposals by what they do when run, not how fluently an AI describes them. Keep scientific objectives, checked against real data or a simulator the system cannot touch, distinct from harness objectives, where the evaluator is software the system may influence. Do not send every parameter through the same generic search: a reliable gradient, an evaluation affordable only ten times, and a need for uncertainty over one best value call for different tools. Record how a candidate failed, not only whether it failed, since a timeout, a constraint violation, and a low score are different diagnoses.
Why it matters now
More companies let AI agents propose executable structures with open numeric parameters, and the evidence on judging them responsibly is thin. An August 2026 method kept structure and parameters separate, routing numeric holes to CMA-ES, variational inference, or NUTS by problem, reporting gains on a small set of hidden-objective studies, not independently replicated. A second August 2026 paper instead let one reasoning agent hold structure and parameters together as a single adaptive search, carrying weaker evidence. A September 2026 study found frontier language models competitive but brittle as direct numerical optimizers, stronger on tasks with more semantic content, an argument for keeping a dedicated solver in the loop.
Ignore the distinction and two costs follow: you pay compute for tuning gains while believing you funded a design improvement, and a harness whose objective doubles as its own scoring software leaves you no defense against a scorer-gaming candidate.
The architecture
I call this the split-and-solve model, four moving parts a leader should be able to name.
The typed sketch comes first: structure fixed in advance, parameters with type, bounds, and units, so proposer and solver read the same document. Second is the solver bench: gradient and constrained solvers for differentiable objectives, zero-order methods for cheap black-box objectives, Bayesian optimization when evaluations are expensive and parameters few, evolutionary methods for noisy objectives, and sampling methods, needing a differentiable density, for uncertainty over parameters rather than one best answer.
Third is the failure ledger. A candidate can fail by not parsing, failing a type check, violating a constraint, timing out, a numerical error, or running cleanly to a low score, six diagnoses one penalty number erases. A timeout is not proof the structure is bad, it is a result censored by budget.
Fourth is the budget-matched verdict. A structure scored after a thousand solver evaluations should never be ranked against one scored after ten. Compare candidates at equal wall time or equal solver calls, and escalate tuning effort under a policy set before the race starts, not after seeing who is ahead.
How to lead it
Ownership belongs with a platform or architecture function, not individual project teams, since results cannot be compared once each team draws its own line between structure and parameters. Fund the typed interface and failure taxonomy first, then a matched solver roster, then a racing policy for escalating tuning on promising candidates early. Measure score against budget spent, not score alone, and track each failure kind and how many structurally distinct candidates survive near-duplicate removal.
Two decisions belong only to an executive: classifying a use case as scientific, checked against an oracle you trust, or as a harness, where the evaluator is software the system can influence, since the two need different proof standards, and authorizing execution of AI-generated candidates at all, which requires resource caps, filesystem isolation, and a defensive posture on what a candidate program can touch, a security decision, not an engineering afterthought.
What it is worth
The savings are in engineering hours and compute, not a headline percentage the dossier does not provide. Routing each parameter to the solver built for its shape, instead of one generic search everywhere, is where wasted compute usually comes from, and a precise failure ledger turns a blind rerun into a targeted fix.
The risk avoided matters as much as the time saved. Mismatched tuning effort can make a mediocre design that got well tuned look better than a stronger one that got a poor solver run, and a harness that scores its own candidates can be gamed by a program that satisfies the scorer without doing anything meaningful.
Measure before and after with a budget-matched score, a feasibility rate, and the rate of each failure kind, not a leaderboard number. The current evidence, a handful of small, differently run studies, none independently replicated, justifies a pilot on one real numeric-optimization decision, not replacing an established optimization process across the business.
Questions leaders ask
- What is the split-and-solve model?
- It is a way of judging any AI-proposed system by separating what stays fixed, the structure, from what gets tuned, the numeric parameters, then routing each parameter to whichever solver actually fits its shape and comparing candidates only after giving each one comparable tuning effort.
- Should we let an AI agent tune its own numbers?
- Only when the objective is checked against something the agent cannot influence, such as real data or a simulator it has no access to. When the agent's own software also scores the result, treat every number it reports as provisional until it is checked by an isolated evaluator.
- How do we know a design change is real and not just extra tuning?
- Compare candidates at equal wall time or equal number of solver calls, never at whatever effort each happened to receive. A structure that reached a high score with ten times the tuning of its rival has not been shown to be the better design, only the better-funded one.
- Is this approach already proven to beat how we optimize today?
- No. The evidence behind it is early, drawn from a handful of small and differently run studies with no independent replication yet. Treat it as worth piloting on one real numeric decision, not as a reason to replace an established optimization process.
- Who inside the company should own this?
- A platform or architecture function, not individual project teams. If every team invents its own line between structure and tuning, none of their results can be compared, and the failure records that make the whole approach useful never accumulate into anything a leader can act on.
