When context windows run short, long documents break. The default fix in the long-context race over the past two years has been "stretch the window to 1M, 10M, even 100M." Bryce Sandlund's arXiv:2610.02404 takes the opposite direction: stop stacking context, and let the model learn to decompose.

Long context, recast as an agent relay

The author shrinks the toolkit to two primitives: read_chunk(start, end) reads any slice of the source, and spawn_subagent(subtask) launches a child whose context is independent. The parent gets an OOLONG-style long document, but every agent in the chain is capped at 8K context. The parent must decide what to read, what to delegate, and when to fold sub-results back together.

This is not prompt engineering. Sandlund's ablations show that base models thrown into the harness perform poorly: base Qwen3.6-35B-A3B, GPT-5.4, and Kimi K2.6 simply forward the original question to subagents without ever reading the document. Opus 4.6 starts to decompose tasks, but burns through $300 of API credits building massive sub-agent trees. Models have no prior for "when to delegate versus read." Pre-training is pure next-token continuation, and agentic post-training does not teach decomposition either. Recursive Language Models-style tools work only because models carry a strong coding prior; on this minimal harness, decomposition has to be taught.

Teaching three decomposition patterns with oracle SFT

Training deliberately picks SFT over RL. The author's reason: the exploration in L-level reinforcement learning cannot escape a weak decomposition prior, and it is unclear what reward to assign sub-agents. GRPO has no obvious group to compare sub-agent trajectories against.

Stronger models (Opus 4.8 / 5 / 5.5 and Fable 5 / 5.1) generate a synthetic dataset that teaches three decomposition patterns:

  • tree reduce for order-independent aggregation tasks (echoing the GPT-3 summarization recipe);
  • sequential fold for order-dependent tasks (the "rolling node" strategy used by Memagent);
  • agentic delegation at boundary conditions, where the model decides whether to read directly, spawn a sub-agent, or fold results.

The key design choice is that decomposition structure is not welded into the harness. The harness only exposes the two tools; how to decompose is left for the model to learn. The fine-tuned backbone is Qwen3.6-35B-A3B, and the oracle data comes from stronger Claude/Gemini-family agents. Code is open-sourced.

8K context ties 1M-token GPT-5.4

Results come in two parts.

On OOLONG-synth, once documents exceed 40K tokens, the 8K-context small model ties GPT-5.4 (medium reasoning, 1M-token context limit). Past a certain length, bigger windows stop buying more long-context capability, while document decomposition starts mattering.

On RULER, across documents from 10K to 320K tokens, the model holds ≥85% accuracy under the per-agent 8K cap. The base model degrades visibly at the long end.

The author emphasizes one point: the training distribution was deliberately kept disjoint from the evaluation distribution. The goal is to show that an SFT'd agent decomposition strategy generalizes to out-of-distribution documents, not to memorize benchmarks.

Why industry has not done this yet

MemGPT (2023), RAPTOR (2024), Chain-of-Agents (2024), and Recursive Language Models (2025) are all "engineered harnesses": humans write the workflow that chops long documents into piecemeal chunks. Memagent (2026) trains decomposition too, yet the strategy stays welded into the harness and only the training distribution is verified.

Sandlund's differentiator: the harness is minimal (two tools), the workflow is not written, and decomposition is fully owned by an SFT-learned policy. The paper also runs ablations showing RL cannot pull it off because exploration is the bottleneck. The only comparable training direction is Kimi K2.5's "agent swarm," but that only trains the parent agent's call policy, not sub-agent trajectories themselves.

Long-context ability is a learnable strategy, not necessarily a bigger window

The most interesting implication: not every long-document problem needs a native 1M context. If the agent learns how to decompose and orchestrate, an 8K window can still carry a 320K document.

The cost: SFT data quality is the ceiling. Whether the oracle decomposes correctly determines whether the strategy generalizes. Training data generation is also expensive (oracles come from Opus 5, Fable 5.1 and similar top-tier models). The paper also admits that "opaque orchestration" trained into the weights may not stay stable outside the training distribution.

Still, for teams that do not ship a 1M window and find MemGPT-style engineering too heavy, this line is worth following: treat context management as a learnable strategy via oracle SFT, instead of always reaching for a bigger native window or an external harness. The next step, which the author flags as future work, is whether an RL stage can plug in on top.