Long-horizon agents have spent the last six months learning to compress their history into summaries. Teams have shipped AutoCompact, CliffCompaction, CompactionRL and a handful of similar systems. They all hit the same wall: a textual summary alone cannot preserve every detail a downstream decision depends on, and the lost information shows up as repeated tool calls, broken web search and broken shell execution. The arXiv paper 2610.11287, posted on October 8, takes a different angle. It trains a small external network to append learned soft memory tokens to the compacted summary, cutting context usage to 5.2% of the original while approaching the full-context score on SummHay.
What REMORY actually does
The pipeline has two steps. First, an external memory network reads the full history and the current summary and outputs a short sequence of continuous soft memory tokens. These tokens are not visible text but continuous vectors that the main LLM receives outside its input embedding layer. Second, the main LLM stays frozen and only sees "summary + soft tokens"; the authors describe this layer as a residual connection along the sequence dimension, where the summary carries the readable narrative and the soft tokens carry the rest.
The trick versus pushing more KV cache into the prompt is that the memory network is trained. Its loss targets the logits of the frozen LLM conditioned on "summary + soft tokens" matching the logits when the same frozen LLM sees the full history. As a result, the main model never needs to be retrained. The team shipped separate memory checkpoints for Qwen3.8-27B and GLM-5.3-Flash: a 1.9B and a 1.24B parameter network, released on Hugging Face as Qwen3.8-27B-REMORY-1.9B and GLM-5.3-Flash-REMORY-1.24B respectively.
What the benchmarks cover
The paper evaluates four benchmarks aimed at long-horizon agents. On SummHay, a summarization benchmark that scores both source attribution and insight coverage, REMORY pushes source attribution up at near-unchanged insight coverage and approaches the full-context joint score using only 5.2% of the input positions. BrowseComp and Terminal-Bench 2.1, the popular web search and shell execution tasks, both see consistent gains on Qwen3.8-27B and GLM-5.3-Flash, with substantially fewer repeated tool outputs and tool errors. The repository ships Codex-style demos where the memory network's compact hook plugs straight into an existing agent harness, and Qwen3.8-27B runs at its default 256K context on two 80 GB GPUs through SGLang.
The training cost is the practical hook. Because the supervision targets the frozen LLM's logits, only the 1B-scale memory network is updated, which fits the budget of most teams that cannot afford to retrain a 27B or larger model.
What this means for the field
Combining textual summarization with learned soft memory uses less memory than KV cache quantization and beats pure text summarization on long-horizon tasks. It is a clean fit for frontier instruction-tuned models built for agents, including Claude, GPT-6 and Qwen3.8-MAX. Structurally REMORY behaves like an external learned long-term memory module, and the parallel to human working memory, where readable recall and implicit indexing are separated, is worth keeping in mind. The obvious next step is to wire this idea into RAG and tool-use training pipelines so small models learn to compress and replay their own working memory instead of forcing the main model to carry long contexts on every turn.