The Update Ledger: Deciding How a Model Should Change
Continual learning, editing and unlearning are three different decisions, and the update ledger decides which one a change actually requires.
Yash Sharma5 min readData, adaptation and alignment
Research area 18: Continual learning, model editing and unlearning engineering
In brief
- Continual learning, model editing and unlearning are three separate decisions, and treating them as one produces models that fail audits.
- The update ledger is a versioned record that classifies, tests and approves every model change before it ships.
- Skipping this testing wastes compute, causes hidden collateral damage and creates legal exposure when suppression is mistaken for removal.
Continual learning, model editing and unlearning engineering answer three questions leadership teams routinely collapse into one: has the model learned something new without losing what it knew, has one fact been corrected without disturbing what is nearby, and has information actually been removed rather than hidden. Confusing them ships a model that passes a demo and fails an audit.
What it is
Continual learning changes a model across a sequence of data or tasks. Model editing targets one localized fact or behavior, leaving the rest untouched in principle. Unlearning claims a model was never trained on specific data, a stronger claim than either. Suppression is different: refusing or lowering the odds of an answer without removing the underlying information, which looks identical to unlearning from outside and is not.
Three metrics get flattened together: stability, how well a model retains what it knew, plasticity, how well it acquires something new, and locality, how narrowly a change stays confined to its target.
Six mechanisms exist, none correct by default: an external source of truth, replaying old data, a modular adapter, editing parameters directly, a dedicated unlearning intervention, or retraining the model. Treating a parameter edit as inherently more rigorous than an external correction is a belief, not a finding.
Why it matters now
Updates increasingly run inside automated pipelines, harder to audit than the models they change. A 2026 comparison of eight update methods on Qwen3-8B, public in July, found fact access, new capability and training efficiency behave as different regimes, not one ranked list. An August 2026 study of spaced repetition during continual pretraining reported improved retention of old and new material in one setting against a uniform-replay control, though the benefit varied elsewhere.
A May 2026 study of recursive edits on two multimodal models reported one model's aggregate edit score rising from 83.66 to 90.73 across 100 edits, while its locality score fell from 96.76 to 89.71 over the same sequence: success in aggregate while leaking into facts never meant to be touched.
A June 2026 study of a representation-selective unlearning method reported a four to forty times stronger reduction in a targeted answer's token probability after a relearning attack, resistance to one attack, not evidence of retraining-equivalent removal. Treating it as proof of removal is how a regulator finds out otherwise.
The architecture
I call the governing tool the update ledger: a versioned record attached to every model change that forces a classification, an evaluation and a rollback plan before anything ships.
The ledger opens by classifying the need: a fresh fact, a new capability, a narrow correction, or a removal requirement carrying legal or safety weight. That decides which mechanism is in scope, and it is where teams most often go wrong, treating a removal requirement as satisfied by an edited weight when only a retraining comparison tests whether the information is gone.
Once a method is chosen, the ledger records the parent model version, the evidence behind the change, and every dataset, weight, adapter and downstream system the change might touch. The candidate then runs a fixed evaluation: the target, related facts nearby, unrelated controls that should not move, and, where removal is the claim, an adversarial attack under a stated threat model.
Updates generally do not commute: a second edit after a first can produce a different model than the reverse order, and undoing the most recent update does not reliably undo an earlier one, especially once a removal requirement is layered in. The ledger tests realistic multi-update sequences directly, rather than certifying each update alone.
How to lead it
Ownership belongs to a named update owner accountable for the ledger, with standing authority to block a release over an unresolved removal claim regardless of which team built it, including vetoing a change that only demonstrates suppression when the requirement was removal.
Fund an evaluation harness covering target, related, unrelated and adversarial cases, provenance tooling that traces a request through every derivative it touches, capacity to compare against a retrained reference, and a validated rollback path. Measure a task-by-update matrix, not one averaged score.
Two decisions do not belong to an engineering team: the smallest improvement worth shipping and largest regression tolerable, a risk-appetite tradeoff, and whether a suppression result with stated uncertainty answers a removal request or demands a stronger retraining comparison, a decision that belongs with an executive.
What it is worth
The money and risk concentrate in three places: compute spent on training or replay that improves an aggregate score while eroding an earlier task, engineering time chasing a local edit's collateral damage, and legal exposure when suppression is presented as proof that information was actually removed.
Measure before and after against the task-by-update matrix the ledger already requires, and compare any claim against two baselines: no update at all, and, where removal is the claim, a model retrained without the affected data. The evidence supports that update methods behave differently across fact, capability and efficiency regimes, that recursive edits can trade locality for aggregate gain, and that a representation-selective unlearning method can meaningfully raise resistance to one class of relearning attack. It does not support a universal ranking of methods, a general retraining-equivalence proof for unlearning, or a return figure. Every number above comes from a single recent study with unreported replication, reason to lean on the ledger's own testing rather than any paper's headline result.
Questions leaders ask
- What is the update ledger, in practical terms?
- It is a versioned record attached to every change a model undergoes that classifies the need as a fresh fact, a new capability, a narrow correction or a removal requirement, then tests the candidate against target, related, unrelated and adversarial cases before it is allowed to publish. A team that cannot produce that record should not be able to promote an update.
- Is suppressing an answer the same as removing it?
- No. A model that lowers the odds of an answer or refuses to give it has suppressed that response without necessarily satisfying any stated removal target. A June 2026 study of a representation-selective unlearning method reported a four to forty times stronger reduction in a targeted answer's token probability after a relearning attack, a real result about resistance to that attack and not proof the information is gone.
- Does replaying old data during training always prevent forgetting?
- Not reliably. A 2026 comparison of eight update methods on Qwen3-8B found that fact access, new capability and training efficiency behave as different regimes rather than one ranked list, and a separate study of spaced replay improved retention in one setting while the benefit varied across the others tested. Treat one positive result as a lead, not a universal answer.
- When should a fact be corrected outside the model instead of inside it?
- Prefer an external, authoritative source when the fact changes often, when provenance has to be demonstrable, or when the correction is narrow enough that retrieval can carry it. Reserve a local parameter edit for cases where its scope and evaluation are genuinely tractable, and reserve retraining for shifts too broad or tangled for either option to audit cleanly.
- Can a bad update just be rolled back?
- Not safely, without checking first. Updates generally do not commute, so restoring an earlier checkpoint can reintroduce information a removal requirement was meant to erase, or silently undo later valid learning. Rollback is a governed operation validated against current requirements, not an automatic undo button.
