Open-Ended Curricula: Why More Environments Can Hurt Training

Open-ended curricula and environment-distribution design decide whether an agent trained on thousands of generated tasks gets more capable, or only busier. Expanding an environment pool by itself has produced nonmonotonic results, and a fixed training budget spread across more tasks gives each one less practice unless the budget grows to match. An executive who funds generation without funding the checks that anchor it to a real deployment objective is paying for volume, not competence.

What it is

Open-ended curriculum design lets a training distribution evolve while an agent learns, rather than training on one fixed set of tasks. Environment-distribution design decides what evolves, how much, and against what standard. Treating every intervention as one difficulty dial hides real differences: what an agent can observe, what actions it can take, what a hint reveals, and how reward is computed are separate levers with separate risks. A reward definition that becomes easier to satisfy changes the objective itself, and reporting that as a more capable learner is a mistake, not a result.

Five principles hold the discipline together. Random or stratified sampling across a representative task pool is a strong baseline, and any adaptive method that cannot beat it has not earned its complexity. A curriculum is a spending policy under one fixed budget, not a ranking of interesting tasks. An environment can only teach what its observation space makes available, so a task requiring information the agent cannot see is unsolvable, not harder. Diversity is counted by distinct mechanisms and required abilities, not task count, since a distribution can grow while adding almost nothing new. And the training distribution is not automatically the deployment distribution, so improvement on one never transfers to the other without a separate check.

Production agents increasingly train on generated environments, and a mistake at this layer produces a system that looks like it learned something when it did not. A study posted in August 2026 that combined environment selection with difficulty interventions across a multimodal agent-learning setup reported the nonmonotonic gains above, an early, author-reported finding not yet independently replicated. The dilution risk is simple arithmetic: a ten-thousand-update budget split across ten abilities gives each a thousand updates, split across a hundred abilities each gets a hundred. Added diversity can still be worth it if it improves shared representations, but that has to be demonstrated, not assumed from environment count alone.

The architecture

The clearest way to run this as one operating model is what I call the anchored mixture model: a fixed deployment objective at the center, an admission gate in front of every generated environment, a mixture split across four protected shares, and a stopping rule that prices every round against real benefit.

The anchor is a protected evaluator and a stable calibration panel, versioned separately from the learner and the generation harness. If all three improve together, a rising score is not identifiable as learning.

The admission gate checks two things before a proposed environment enters training: whether the task is solvable at all using a privileged reference solution, and whether it is still solvable using only the observations the agent actually has. A task where two situations look identical from the agent's own observation set is a coin flip dressed up as a challenge, no matter how much training follows. Environments that fail either check are rejected, but their failure reason is archived so the next generation round does not repeat the same dead end.

Everything that passes lands in one of four protected shares: deployment-mirroring tasks, replay of mastered skills, learnable frontier tasks, and rare but consequential safety scenarios. The shares are fixed in advance, and only the frontier and replay allocations adapt as training proceeds. A generator left free to redefine what counts as success, rather than only the circumstances an agent faces, has stopped doing curriculum design and started doing objective design, a decision that belongs to people, not the generator.

Ownership splits on purpose: the team accountable for the deployment outcome owns the objective and calibration panel, frozen before curriculum work begins, since changing what counts as success mid-program is a business decision. A separate function owns the generator, the admission checks and the archive, so no group can both claim progress and grade it. Fund the anchor and the gate before more generation compute, and count full economics, including every failed generation. Two decisions belong to the executive alone: freezing or changing the deployment objective, and setting the stopping bar that justifies one more round of generation.

What it is worth

The money and the risk show up in three places: compute spent training against diluted or impossible tasks, cost that never gets counted because it lives in failed generations rather than the training run, and exposure created when a generator exploits its own simulator rather than a weakness that would matter in production.

Measure before and after with the same yardstick: deployment-weighted transfer against a frozen panel, replay retention, safety coverage and invalid-task rate. The 2026 evidence supports funding a protected evaluator, an admission gate and cost-matched comparisons. It does not support a universal claim that open-ended generation beats a well-built fixed or stratified mixture. Failing to beat that mixture, at matched cost, is itself the evidence against continuing to expand. No dollar return appears anywhere in the underlying evidence, and any figure attached to it goes beyond what the record shows.