arXiv 2605.15156 introduces MeMo (Memory-Module), a method for decoupling the "memory" and "reasoning" capabilities of LLMs. The result: knowledge can be updated in MeMo-equipped LLMs by simply swapping the memory module, with no retraining and no quality loss.
The "memory-reasoning coupling" problem: in standard LLMs, knowledge is "baked into" the model weights. To update a fact (e.g., "the current US president is X"), you need to fine-tune the model, which is expensive and can degrade other capabilities.
The MeMo fix: a separate "memory module" that stores factual knowledge as a set of (key, value) pairs. The memory module is queried at inference time, and the result is fed into the LLM as additional context. The LLM itself is responsible for "reasoning" over the queried facts, not for "remembering" them.
The benchmark: MeMo-equipped LLMs match the quality of fully-trained LLMs on factual QA (NaturalQuestions, TriviaQA), while allowing 100× faster knowledge updates. A knowledge update that would take 100 GPU-hours with fine-tuning takes 1 GPU-hour with MeMo.
The "no retraining" highlight: the most significant benefit of MeMo is that knowledge updates don't require retraining. This means: (1) real-time knowledge updates (e.g., news, stock prices) are possible; (2) personalization (per-user memory) is practical; (3) "knowledge editing" (fixing model errors) is fast.
The bigger takeaway: "decoupled memory" is the right architecture for knowledge-intensive LLMs. The "bake everything into weights" approach is too inflexible, and the "memory module" approach gives a clean separation of concerns. For the industry, this means "knowledge base + LLM" architectures will become standard, with the knowledge base being updatable in real time.