For the past two years, agentic memory has been one of the hottest directions in AI engineering. Every framework races on the same thing: pull the information out of long conversation histories, organize it, index it, store it as structured memory, and let the agent retrieve it on demand. Mem0, MemMachine, Memora, EverMemOS — the names sound more and more like products, and the architecture diagrams get more and more elaborate.

But a paper posted to arXiv on October 22 invited all of this into a test track, ran it around, and reached a conclusion that stings: the most naive RAG setup won almost everywhere.

The paper is "Is Complex Agentic Memory Always Needed? Naive RAG Is Stronger Than We Thought" (arXiv:2610.30755), by Zhengye Bao, Shengtao Zhang, Haokai Zhang, Shiqi Lou, Muning Wen, and Minnan Luo at Xi'an Jiaotong University, 62 pages in total. The authors named their baseline Raw-RAG: no extraction, no structural organization, no LLM rewriting of history — just serialize the raw conversation with timestamps, chunk it, run dense retrieval plus reranking, and hand the evidence straight to a reader model.

How much it won

Evaluation covers four benchmarks: LoCoMo (10 conversations, 1,540 non-adversarial questions), LongMemEval (500 examples), MEME (100 episodes, 1,188 phase-specific questions), and MemoryAgentBench (14 sources, 2,071 questions). The comparison set spans 12 memory systems on the LoCoMo and LongMemEval side and 7 on the MEME and MemoryAgentBench side, including Mem0, A-MEM, MemMachine, Memora, EverMemOS, EMem, SimpleMem, MemOS, MemU, Nemori, RecMem, and MemoryOS.

The result: across seven benchmark-model settings, Raw-RAG takes six of them, losing only on MEME with a DeepSeek-V4-Flash reader, where MemMachine wins. For the exact margins, the official GitHub repository publishes the comparison table: on LoCoMo, 89.07 with a GPT-4o-mini reader against MemMachine at 88.12, and 93.70 with a GPT-4.1-mini reader against EverMemOS at 93.05; on LongMemEval, 87.53 against Memora at 87.40; on MemoryAgentBench the gap widens to 48.32 against Memora at 43.84 with GPT-4o-mini, and 53.41 against 47.88 with DeepSeek-V4-Flash. The abstract puts the range as leading the strongest compared system by 0.13 to 5.53 points.

What is notable is that the LongMemEval margin is only 0.13 points — essentially a photo finish. The paper itself does not dress this up as a rout.

The sharper result is the counter-evidence

If it were merely "the simple method can also tie", this would be one more parity paper. What actually hurts is the three experiments that follow.

The first audits Mem0 write decisions. Across 33,902 write decisions on LoCoMo, UPDATE and DELETE together account for only 4.39%; across 428,858 write decisions on LongMemEval, only 2.46%. Effective mutations — ones that actually changed text or deleted content — come to just 1.90% and 1.34%. The authors then swapped UPDATE for append-only ADD and DELETE for NONE, and LoCoMo accuracy went up, from 67.36% to 69.42%. In this configuration, explicit memory rewriting bought no accuracy at all.

The second adds memory machinery to Raw-RAG. All three variants get worse in the same MemoryAgentBench setup: atomic fact extraction drops to 28.70 (19.62 below the 48.32 baseline), summary augmentation loses 5.81 points, and graph expansion loses 5.78. The paper explains this by noting that atomic fact extraction takes test-time learning from 47.68% down to 15.37%, because the structure of usable demonstrations is broken apart.

The third is cost. In the paper latency table, Raw-RAG finishes ingestion plus retrieval in 305.91 seconds on the same 10-episode, 110-question MEME slice; EMem takes 1,561.97 seconds, MemOS 2,769.56, MemMachine 3,515.74, Mem0 4,142.95, and EverMemOS 9,301.48. The paper states this is 5.1 times faster than the next system and 30.4 times faster than EverMemOS. And adding those three memory-management operations costs 21.6 to 109.3 million LLM tokens and 2.9 to 8.8 hours, where Raw-RAG needs 12 minutes for the same work.

Why

The mechanism the paper offers rests on two metrics: coverage and utilization. Coverage is whether the retrieved evidence is sufficient to answer the question; utilization is whether the answer comes out right once the evidence is there. On MemoryAgentBench, coverage spans 19.07% to 45.29% across systems, while utilization spans only 80.00% to 89.71%. The conclusion is that what separates systems is not how well they read evidence once they have it, but whether the needed evidence is supplied at all.

The paper also introduces path coverage to explain long-chain questions: on MEME tracking and cascade items, Raw-RAG reaches 62.34% conditional path coverage, MemMachine 56.70%, and MemU only 9.62%. Retrieving the answer value is one thing; also preserving the temporal relations and dependency chain needed to derive it is another, and that is exactly where memory systems lose points.

The sharpest blade is the complete-store comparison. The authors run three conditions: each system native retrieval, a complete-store oracle that uses ground truth to select all phase-visible memories, and a raw-evidence control that also uses ground truth but picks the original conversation segments. The result is that Raw-RAG and MemMachine, both of which retain raw conversational content in their evaluated stores, recover to 57.54% and 57.46%, while the raw evidence itself scores 61.60%. Even granting full access to processed memory stores does not close the gap to raw evidence. With the packing ceiling removed, all seven systems still sit 7.58 to 29.38 points below raw evidence, and every paired 95% confidence interval excludes zero.

The paper states it plainly: information dropped or distorted at ingestion time cannot be recovered through retrieval.

Who should worry

This is not a paper saying memory systems are useless, and the wording is carefully restrained. It draws three boundaries: the evidence holds only for offline conversation tasks on public benchmarks and cannot be extrapolated to interactive, multimodal, or real production settings; raw storage grows linearly with history length and raises storage and privacy concerns; and state reconstruction quality still depends on reader model capability. In the appendix, on PersonaMem-v2, Raw-RAG scores 49.62%, ranking third out of seven behind MemOS and EverMemOS, and leading no individual scenario.

But the acceptance bar it proposes is concrete. The paper says, in its own words, that improved memory systems should demonstrate gains over a well-specified naive RAG baseline and establish what the added complexity contributes. Translated into plain language: if you are building a memory system, the next benchmark table must contain a row for the complexity you cannot beat with naive RAG.

The community response is worth noting. The repository README carries a dedicated section thanking the GoodMem team for an independent reproduction: running LoCoMo with the same embedding model, reranker, and retrieval depth, they got 88.68 with a GPT-4o-mini reader and 93.48 with a GPT-4.1-mini reader, against the paper reported 89.07 and 93.70, both inside the 95% confidence intervals. That reproduction also shows up in the comments on the HuggingFace paper page.

So what

The value of this paper may not lie in "naive RAG won" but in putting a long-unexamined premise on the table: is the complexity of agentic memory actually demanded by the problem, or by industry consensus?

The authors are fairly measured about it. They are not saying you should not build memory systems. They are saying that if you do, your ablations have to include a naive RAG baseline, and you have to report end-to-end cost — ingestion plus retrieval — not just accuracy. The gap between EverMemOS at 9,301 seconds and Raw-RAG at 305 seconds says more about the price of an architectural choice than any benchmark score does.

One thing the paper does not prove is worth flagging: everything it tests is "given an existing history, how do you answer questions". The other half of the memory problem in real products is when to remember, what to remember, when to forget, and what to do when a data subject asks for deletion. Those are exactly the things structured memory systems are selling, and exactly the things naive RAG cannot help with. The limitations section does touch on privacy and deletion constraints, just without going deep.

So the more accurate reading is not "memory systems are useless", but rather: until you prove you are stronger than "throw the raw conversation into a vector database with a reranker", a complex memory architecture is probably paying for requirements you imagined.