What if you inject the same total amount of noise into a large language model, but change only where it lands? A paper submitted to arXiv on 10 November (arXiv:2611.14209) quantifies the gap: identical noise budget, applied randomly versus applied in concentrated bursts across consecutive tokens, and the resulting GSM8K accuracy differs by 12.4 percentage points. The author is Karol Struniawski of the Institute of Information Technology at the Warsaw University of Life Sciences (SGGW).
The setup: same budget, two delivery schedules
The framing comes from neuroscience. During extended wakefulness, mammalian neocortical networks spontaneously exhibit localized periods of transient neuronal silence, called off-states or microsleeps. In humans, these intrusions predict cognitive lapses during tasks that demand continuous, sequential attention. The question the paper asks is analogous: an autoregressive model's reasoning chain unfolds token by token, and the output at step t conditions every later step. Does a spatially and temporally concentrated structural disruption therefore hurt more than an equal cumulative budget of diffuse, independent noise?
The evaluation uses 250 paired GSM8K derivations across two 8B open-weight models, Qwen3-8B and LLaMA-3.1-8B-Instruct. Masking targets only the upper half of the network: layers 18 through 35 in Qwen (576 vulnerable heads) and layers 16 through 31 in LLaMA (512 heads), both using grouped-query attention with 32 query heads sharing 8 key-value head pairs. Two schedules are compared. In the burst arm, masks are stacked on consecutive decoding steps. In the i.i.d. control, the same cumulative budget is dispersed across the whole generation horizon via stratified quintile matching, which guarantees matched marginal temporal distributions between conditions.
Where the 12.4 points come from
At onset parameter B=0.10, which schedules roughly 5.7% of the vulnerable head-token volume, the two arms separate clearly.
For Qwen3-8B, the clean baseline is 93.6% (234/250). Under diffuse i.i.d. dropout it is 91.2%; under burst dropout it falls to 78.8%. That is a paired deficit of 12.4 percentage points relative to diffuse noise, with exact McNemar p=5.5e-6, Holm-corrected p=1.1e-5, and a 95% bootstrap confidence interval of [-17.6, -7.2].
LLaMA-3.1-8B-Instruct starts from a clean baseline of 87.2%. Under i.i.d. noise it scores 84.0%, and under burst dropout 76.8%, a gap of -7.2 points with p=0.0198.
Crucially, the realized budgets are matched: on Qwen the burst and i.i.d. arms consume 8,897.3 and 8,631.3 head-tokens respectively, and on LLaMA 6,626.4 against 6,437.2. So what differs is not how much noise was delivered but how concentrated it was.
A second layer of analysis asks whether the damage comes from instantaneous per-step density or from multi-token temporal continuity. In an exploratory factorial experiment on 100 paired derivations, "spaced" single-step-dense dropout already costs Qwen 10.0 points relative to i.i.d. (Holm-corrected p=0.0425). Chaining masks across three consecutive steps adds only another 7.0 points, which does not survive correction (p=0.1892, confidence interval spanning zero). The evidence therefore points more toward a sudden large hole in a single step than toward continuity across steps.
Why concentration hurts: a KV cache you cannot roll back
The paper proposes an interpretive model called the Autoregressive Attractor Trap. When an attention head is masked at some layer, the residual stream is perturbed; the Key and Value vectors of all subsequent layers are then computed from that corrupted representation, and an autoregressive decoder permanently appends both the emitted tokens and the KV vectors to its context. Once attention capacity is restored, the surviving heads still read the dirty cache, and the decoding trajectory is trapped in an erroneous attractor basin from which greedy decoding cannot escape.
The author is careful here, explicitly framing this as an interpretative model rather than a proven causal theorem, because no surgical KV-cache state rollback was performed. The paper also concedes that part of LLaMA's deficit is formatting collapse rather than reasoning error: 8 of 36 discordant losses (22.2%) never emitted a final answer line. Evaluating accuracy strictly conditional on emitting an answer line narrows LLaMA's gap from -7.2 to -5.0 points. Qwen behaves in the opposite way, with conditional accuracy falling from 91.9% to 80.5% (-11.4 points), meaning its failures are largely wrong derivations rather than missing formatting.
Three disclosures worth reading on their own
A single-author preprint on a small sample earns trust not by producing beautiful numbers but by writing down its own holes.
First, not all of its statistical thresholds were met. The pre-registered significance level was alpha=0.01, while LLaMA at B=0.10 reached p=0.0198, which does not clear it, and the author lists this plainly. LLaMA's answer-format omission rate of 9.6% also exceeds the registered 2% threshold, and so do the i.i.d. control arms (4.0% and 6.0%).
Second, single-draw variance was underestimated. On the same 100 items under the confirmatory schedule, the burst-minus-i.i.d. contrast was -6.0 points on Qwen and -5.0 on LLaMA, whereas the exploratory run's random schedule yielded -17.0 and -2.0. The author therefore states that the -10.0 point single-step density effect must be read strictly as an exploratory, draw-specific finding. On LLaMA the reference contrast failed to replicate altogether (-2.0 points, p=0.8145), so the tie between the Spaced and Burst arms cannot be interpreted as mechanistic invariance.
Third, the high-dose regime is non-monotonic. At B=0.15 Qwen loses 23.3 points, but at B=0.20 the loss shrinks to 15.0 points. That exploratory sweep ran on only 60 items, and the author declines to read it as a pattern because it cannot be distinguished from schedule-draw stochasticity.
The B=0.05 arm is worth flagging too: for Qwen the bootstrap confidence interval excludes zero, while the exact McNemar test yields p=0.0614 and does not. The author explains the divergence, since the percentile bootstrap resamples paired item differences with replacement whereas the exact McNemar test evaluates a binomial probability on the 29 discordant pairs.
So what
The value of this paper is not the number 12.4 itself. It is that it turns an engineering intuition into something measurable: the fragility of an autoregressive reasoning chain comes from structure, not from total volume.
That changes how "add noise and test robustness" should be designed. Numbers obtained by scattering masks randomly describe a different fault mode than the occasional, local, consecutive head failures that occur in the wild. The first measures average resilience; the second is what actually breaks a reasoning chain. Any evaluation that treats "random dropout costs only two points" as a robustness metric is measuring a fault that does not exist.
There is a second implication for engineering practice: reversibility of the KV cache may deserve more investment than raw model accuracy. The paper failed to verify the attractor trap precisely because it performed no cache rollback, and yet "can we recover the contaminated stretch of context" is exactly the question long-context agents hit repeatedly in production, where one failed tool call leaves dozens of subsequent reasoning steps built on a dirty state.
As for the neuroscience analogy, the author is explicit that it is an operational parallel and not a physiological equivalence. Attention heads do dot-product associative routing over token vectors, which is not synaptic transmission and spiking dynamics in cortical columns. What is worth keeping is the underlying phenomenon: local silence interrupts continuous cognition in the human brain as well. Whether an LLM's own "microsleeps" can be detected and repaired may turn out to be a more interesting question than squeezing out another point of benchmark accuracy.