Stuffing an entire book into the context window and then hoping the model remembers what mattered is the default approach to long-document QA — and also the most expensive one. A team from Columbia University suggests the real problem is not window size, but that nobody ever trained models to take notes.

The paper is "Memento: Learning Context Management for Long Context Processing Agents", posted to arXiv on 30 October 2026 (arXiv:2611.00244). The task is specific: multi-hop question answering over whole books. In their data, 90% of books exceed 51,000 Qwen3 tokens, 70% exceed 90,000, and the longest reaches 2.5 million tokens. The models they train with have context windows of only 16K to 20K — meaning a competent agent has to carry its own progress across dozens of windows.

The problem is bad notes, not long context

The paper makes an uncomfortable observation: off-the-shelf agentic models do not know how to take notes. GPT-5.5 wrote not a single compaction note across a 337-sample test set. Across 12,000 GLM-5.3 rollouts on the training set, not one contained a valid note. Qwen3-8B did slightly better, but it tends to write identical notes back to back, and only about 6% of its trajectories are valid.

Worse, reinforcement learning alone cannot produce the behavior. The reason is exploration: if the policy never samples a good note, RL has nothing to reinforce. The paper reports that without SFT warm-starting, the compaction rate collapses to near zero within the first 20 RL steps, and the policy abandons the skill entirely.

Two-stage training

The fix is to demonstrate the behavior with synthetic data first, then use RL to make it reliable.

Step one is SFT cold-start. The team ran GLM-5.3 in their harness to collect correct trajectories, then injected compaction notes after the fact — asking the model to summarize key findings, mistakes to avoid, and next steps. Notes that failed the validity heuristics were resampled; search actions and final answers were left untouched. Roughly 6K trajectories survived filtering and were used to fine-tune Qwen3-8B.

Step two is RL, using GRPO, with a reward built from four parts: format compliance (which gates everything else — no format, no reward), note validity, answer correctness, and confidence calibration via a negated Brier score. Note validity rests on three hand-crafted heuristics: every non-final window contains at least one note and the rollout has fewer than five in total; no note may be the first tool call in its window nor immediately follow another note; and no two notes may exceed 80% token-set similarity.

The key design is decoupling training from inference. Training only ever exercises "compact into a fresh window," yet the same note structure supports two more strategies at test time: context folding (also retaining the turns after the latest note) and confidence-based backtracking (reverting to an earlier note checkpoint after a low-confidence answer and generating an alternative continuation).

The numbers

Training and evaluation used LiteraryQA, NovelHopQA, and the English subset of SagaScale, with NovelQA held out entirely for generalization testing — 4.7K test questions in total. The original Qwen3-8B agent scored 36.9%; the full system reaches 67.8%, while never holding more than 20K tokens in context.

The comparison numbers are starker. The paper reports that GLM-5.3 at 753B scores 69.3% in the same harness at roughly 9x Memento's cost, and that placing the whole book in GLM-5.3's context window yields 74.3%, 6.4 points higher, but at more than 26x the inference cost. GPT-5.5 reaches only 13.1% in the same harness — though the paper supplies the reason: only 21.4% of its rollouts produce a parsable final answer, and it never calls the note tool, so even after correcting for output format it lands at 57.1%, at 54x the cost. Worth flagging: GPT-5.5 was evaluated on at most 100 questions per dataset because of API cost, and the SagaScale test split contains only 37 questions, so the noise is not trivial.

The ablations show why both stages are needed. With a single window, the untrained model gets 29.6%, SFT gets 34.4%, and SFT+RL gets 58.6%. Adding context folding: 32.6% / 46.6% / 66.9%. Adding confidence-based backtracking: 36.9% / 59.1% / 67.8%. SFT alone is insufficient because it over-compacts and accumulates errors across windows; RL alone is insufficient because it never samples anything worth reinforcing.

My read

The genuinely interesting part is not "an 8B model beats GPT-5.5." That comparison depends heavily on the harness — GPT-5.5 scores low mainly because it was never taught to use this note-taking tool, and the paper says so in its own text.

Two things are worth keeping. First, context management is not prompt engineering; it is a trainable capability. The prevailing assumption is that long context gets solved by buying a bigger window, but this paper's cost curve says otherwise: buying the window costs 26x, learning to compress costs a ninth. For long-document retrieval or codebase comprehension, where an agent re-reads large volumes of material, that is an easy calculation.

Second, ordering matters more than recipe. RL's failure mode here is instructive — not that it trains poorly, but that it cannot train at all, because its exploration space contains no reinforceable samples. SFT does not primarily raise accuracy; the paper notes that at the start of RL, validation accuracy is nearly identical with and without SFT initialization. Its job is to push the policy into a region where RL can sample good trajectories. That is a general lesson for any project trying to teach RL a behavior the base model does not have.

The boundaries are real, too: English-only whole-book QA, short answers, LLM-judge grading; note validity enforced by hand-crafted heuristics that may not capture usefulness; and training covers only compaction into a fresh window, with folding and backtracking existing purely as test-time transfers. What it demonstrates is that this route works, not that it is solved.

The context-window arms race may be quietly getting routed around by a different approach: not letting models hold more, but teaching them to throw most of it away.