Every time an LLM agent calls a search tool, the returned page content can easily run to thousands of tokens, while the next decision may hinge on a single name or a single year. A paper posted this week on arXiv:2611.06500 by Tsinghua's AI Agent Group starts with an accounting exercise: by the fourth tool call, raw tool observations already occupy 96.7% of the agent's context. These long, noisy observations not only burn money but also dilute the model's attention to key evidence. Their proposal, Coca-Cola (Context Compression via Continuous Latent Tokens), takes an aggressive stance: tool outputs do not have to be human-readable text at all.

The hidden ceiling of existing compression

Mainstream context compression falls into two camps. Token-pruning methods (the LLMLingua family) have a model score token informativeness and drop the rest. Text-summarization methods (RECOMP, Search-o1's Reason-in-Documents module, ACON) use an auxiliary LLM to condense tool outputs into shorter prose. The paper's critique applies to both: the compressed result is still a discrete token sequence that must remain semantically coherent, which inherently caps the compression ratio. Continuous latent tokens escape this constraint — a single vector can jointly encode information across many numerical dimensions, and the paper formalizes this intuition with a codebook-capacity argument showing latent representations can carry more information per input position than a vocabulary symbol.

128 latent tokens replace a whole observation

Coca-Cola pairs a small encoder with a latent adapter. A compressor model — Qwen3.5-4B or GLM-Z1-9B — jointly encodes the task instruction, the agent's current reasoning trajectory, and the tool observation. A latent adapter then maps the hidden states into a fixed number of latent tokens placed directly into the target agent's input embedding sequence, delimited by special boundary markers. At inference, every tool observation is compressed into exactly 128 latent tokens; in the web-search setting the agent retrieves up to 5 results per call and makes at most 5 tool calls per question. Target agents are Qwen3.5-35B-A3B and GLM-4.7-Flash.

One training objective: do not change the next action

The core design is Action-Preserving Training. Instead of reconstructing the original text, the compressor feeds the full observation and its latent-token version separately through the same frozen agent, aligning the two next-action distributions with KL divergence (loss weight 0.3), plus cross-entropy supervision from the recorded next action in the trajectory (weight 1.0). Because latent tokens are continuous and differentiable, gradients propagate through the frozen agent back into the compressor; only the encoder's LoRA parameters (rank 16) and the adapter are updated. Training uses 3,927 pairs (436 held out for validation) on 4 A100-80G GPUs — 3 GPUs host the frozen agent, 1 hosts the compressor — with learning rate 1.5e-4 linearly decaying to 5e-5 and the KL term warming up over the first 20% of steps.

Up to 89% fewer tokens, and scores go up

Across NQ, TriviaQA, 2WikiMultiHopQA, and HotpotQA with two target models, the efficiency gains are decisive. Input tokens drop from 14,362 to 1,766 on NQ with the Qwen agent (down 87.7%), and from 31,693 to 3,003 on 2Wiki with the GLM agent (down 90.5%). End-to-end runtime falls by up to 56.0% (GLM on 2Wiki, 106 s to 47 s). Overall compression rates land between 70.01% and 89.24%. The counterintuitive part is quality: after compression, EM and F1 consistently beat the uncompressed Vanilla baseline. On HotpotQA with the Qwen agent, EM rises from 11.00 to 25.70 and F1 from 16.41 to 33.75; on TriviaQA with the GLM agent, EM rises from 41.50 to 51.50. Coca-Cola takes the best EM and F1 among all methods in six of eight model-dataset settings. There is an honest exception: on TriviaQA with the Qwen agent, ICAE — trained with a text-reconstruction objective — scores higher on EM (61.80 vs 59.20), and the paper does not hide this.

So what

The paper states the compressor has no search-specific design; swapping in another tool requires no architectural change. That means "tool outputs as latent representations" could become a general-purpose interface layer. My take: context is the most expensive resource an agent has, and most work optimizes what gets inserted into it — this paper reframes the question as "must the things in context be text at all?", a bigger conceptual step than most compression work. The costs are equally clear: latent tokens are unreadable to humans, making debugging a painful affair when the agent misbehaves, and the compressor needs its own GPU. The authors claim (to their knowledge) to be the first to compress tool outputs into latent tokens. The direction is worth watching: if tool-protocol layers ever start emitting latent representations natively, the entire cost structure of agents gets rewritten. Code is on GitHub: THU-Agent/Coca-cola, with a Chinese README providing reproduction paths for the Vanilla, LLMSummary, and LLMLingua baselines, Exa as the default web-search backend, and full training and evaluation scripts.