Context for language models has long had only two compression routes: have another model write a human-readable summary, or render documents as images and OCR them back. Both restore information to "text" — yet the consumer of that compressed result is not a human, it is a model. LatentPress, from Zhengze Zhou (Cornell University) and Hejian Sang (Iowa State University), takes a third path: conversational histories and long documents are written directly into continuous memory tokens that a frozen decoder reads through its input-embedding interface, with no text reconstruction at inference (arXiv:2609.01507).

Training only 0.1% of the parameters

LatentPress splits context use into WRITE and READ: WRITE maps text or a document into a compact continuous state; READ feeds that state straight into the frozen decoder's input-embedding layer. The only component trained is a small reader-matched writer adapter, 4.2M to 26.2M parameters — about 0.1% of the decoder. Compression rates span 4x to 16x. Compared with prior routes: Gist fine-tunes the whole decoder, ICAE trains an LLM-scale encoder and still reconstructs through an autoencoder — LatentPress's pitch is "train only an adapter, never reconstruct, read soft tokens directly."

Summarization is close to a disaster

On the 500 oracle-evidence questions of LongMemEval (judged by Llama-3.1-70B-Instruct, Qwen2.5-7B reader): uncompressed evidence scores 0.490; LatentPress reaches 0.504 at 7.70x compression, with 0.476 at 4.62x and 0.478 at 6.27x — compress harder, score higher. The baselines fare badly: text summarization collapses to 0.184 at 12.06x; the DeepSeek-OCR route slides from 0.426 at 2.33x to 0.312 at 9.34x; ICAE crashes to 0.174 at 17.28x.

On LongBench-QA, in-domain-trained writers beat reading raw text across the board at 4x compression: Qwen2.5-14B rises from 47.93 to 57.99, Qwen2.5-7B from 43.80 to 49.06, Qwen3-8B from 30.80 to 39.62. At 16x, however, all three readers fall below raw — the compression dividend has a boundary.

43ms writes, 5-9x faster reads

Writing one conversation takes 43 ms (Qwen3-8B, H100 80GB), versus roughly 934 ms for batched DeepSeek-OCR reconstruction and 407-645 ms for text summarization — an order of magnitude slower. On the read side, a compressed prefix is 5.0-9.2x faster than raw context: Qwen2.5-14B takes 4.14 s on raw, LatentPress only 0.49 s.

Where the cold water goes

Cross-backbone generalization is reader-dependent: a writer trained on UltraChat transfers zero-shot to LongMemEval, matching raw on Qwen2.5-7B, exceeding raw on the weaker Qwen3-1.7B, but staying below the uncompressed baseline on the stronger Qwen3-8B. The repository currently has 2 stars, ships no model weights (zero models on Hugging Face cite the paper), and reproduction runs through Docker scripts validated on B200. Judging relies on a single judge — its preferences are worth keeping in mind.

So what

The paper's real jab is at the interface: the industry defaults to compressed context being "readable text," when the thing consuming it is a model. The 0.184-vs-0.504 gap between summaries and soft tokens shows that the "rebuild it for humans" step is both slow and lossy; the 16x collapse reminds that continuous memory's sweet spot sits at 4-8x. The next question worth watching is who first wires a soft-token interface into a production agent's memory layer — the paper ships code and reproduction scripts, so the ball is in engineering's court.