The Reach-Record-Prove Method for Attention Architecture Decisions

Attention topology, locality and positional-state engineering determines which tokens a model can actually see and whether the record of where each token sits stays correct as a conversation runs long. A widely discussed 2026 retrofit comparison found that a plain window of 64 tokens plus four fixed anchor tokens kept roughly 99 percent of a full-attention teacher's average score across six benchmarks, with no additional training. That number is real, and it is exactly the kind of average an executive should not fund a production switch on before asking what the missing one percent contains.

What it is

Attention topology is the graph of which tokens a query is allowed to see. A mask drops certain query-key pairs, answering an exact, narrower question rather than approximating the full one. A kernel or linear feature map replaces the scoring computation itself, often compressing an entire history into a fixed-size running state. Sinks keep a handful of fixed prefix positions permanently visible regardless of the window's position, and global tokens and landmarks add selected long-range connections on an otherwise local pattern.

Locality and reach are not the same property. Stack enough causal layers with a local window and they compose into a much longer effective reach, since a fact can hop layer to layer across the sequence. That composed reach is a connectivity bound, which paths exist, not a guarantee a fact survives the compression along the way, so retrieval at long range has to be tested directly, not inferred from the geometry.

Positional state is four separate quantities collapsed into one number by habit: logical position, physical cache slot, segment identity, and, for rotary encodings, phase. Evicting an old cache entry does not make its replacement the same age. Reindexing only one side of a rotation, queries but not retained keys, can silently corrupt every score touching that key. Full causal softmax attention is the baseline every shortcut trades access against, for lower memory or compute.

Two 2026 studies show why. The retrofit comparison above, made public August 28, 2026, is graded B for inspectable evidence with bounded limitations, and by its own account that average can conceal critical long-range failures and says nothing about performance trained from scratch. A second study made public August 6, 2026, converted six of twenty attention layers in a 16-billion-parameter diffusion model to a block-hybrid pattern, exact attention inside the active block, linearizing the rest, graded C, preliminary and abstract-limited, answering a narrower question about diffusion-style generation. Neither result establishes one topology as correct across scratch training, retrofits, diffusion models and agentic workloads.

The architecture

I organize every topology decision I am asked to fund around the reach-record-prove framework: the two studies above each fail differently when a team skips one of its three steps.

Reach comes first: choose the topology by the paths the task needs. A support agent answering from the last few turns needs little reach, so a window-plus-sink pattern is enough. A contract-review system finding one clause among thousands of tokens needs guaranteed long-range access regardless of its aggregate benchmark score.

Record comes second: track the four positional fields separately. Update position metadata atomically on every cache eviction, so a replacement occupying an old physical slot never inherits that slot's logical age, and apply rotational encodings consistently to both sides of a query-key comparison, since reindexing only new queries while retained keys stay on the old phase corrupts scores without raising any visible error.

Prove comes third: validate with equal-context-access and equal-runtime comparisons, evidence placed outside the window and among similar-looking distractors. Report a distance-conditioned curve, accuracy as a function of how far the needed fact sits from the query, rather than a single mean, since a high average is what let the 99 percent retrofit result look complete while hiding long-range gaps. The clearest failure is treating one step as answered by the others.

Ownership sits with a named attention-topology and position-state owner, funding a deployment-shaped test panel before any topology change ships and latency and memory measurement at real batch sizes, since attention FLOPs alone omit projections, feedforward blocks and data movement, and a sparse layout can lose enough utilization to erase its advantage.

Two decisions belong to an executive alone: how much long-range retrieval risk to accept for the memory and speed a restricted topology buys, since a score close to a full-attention teacher can still hide a costly rare failure, and when a position contract must be re-validated, such as after a model swap or a cache-format change, rather than assumed to still hold.

What it is worth

A concealed long-range failure reaches customers when a team ships on an aggregate score that looked complete, the risk the 99 percent retrofit comparison warns sits inside its own average. Engineering time is lost chasing corrupted output after an eviction silently shifts a cached key's effective position, a failure atomic bookkeeping prevents. A sparse or linear topology chosen for its FLOPs advantage can still cost more once data movement and accelerator utilization are included.

The 2026 evidence supports that a well-tuned window-plus-sink retrofit can match a full-attention teacher closely on average, in a bounded comparison, and that a block-hybrid pattern can serve diffusion generation with exact local attention. It does not support a single winning topology across scratch training, retrofits, diffusion models and agentic workloads, and no return percentage should be assumed from either study.