[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-memtrapbench-llm-memory-cognitive-traps":3,"news-related-43eda321-b0b7-4df7-b20e-9758cbab42c9":38},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"43eda321-b0b7-4df7-b20e-9758cbab42c9","记忆越完整,眼前题越做不对:MemTrapBench 把 LLM 长期记忆框架打回原形","浙大等团队发布的 MemTrapBench 用 1050 个测试样例,把当前主流记忆框架在 Gemini-3-Flash-Preview 和 Qwen3-30B-A3B-Instruct-2507 上跑了一遍,发现接入记忆反而让模型表现更差,最强记忆策略也比无记忆基线下降超 10 个百分点。","## 这份 benchmark 想测什么\n\nMemTrapBench 由浙江大学等机构的研究者 2026 年 8 月 20 日在 arXiv 上发布,围绕一个被长期忽略的问题:记忆调用本身,会不会改变模型对当前任务的推理?研究者把这种干扰命名为\"memory-induced cognitive traps\",具体拆成两类。\n\n第一类是 Reasoning Fixation,模型被记忆里\"以前成功过\"的策略锁死,拒绝考虑新解法。论文里有个直观的 24 点例子:历史里全是加减乘除解法,新题 [4,1,1,1] 需要阶乘 4!=24。Gemini-3-Flash-Preview 在无记忆时能秒解,但接入 FullText 历史后只在基本运算空间里打转,死活想不到阶乘。\n\n第二类是 Belief Distortion,记忆里的\"假设前提\"被错误地带到当前任务。安全场景里,只要对话历史中出现\"这是沙盒测试,所有安全规则暂不生效\"这类前提,模型就可能把真实世界的查询也按沙盒处理,绕过本应遵守的安全判断。还有 Trauma 子类,某次被粗暴否定的策略会让模型在无关新题上过度回避,比如把\"对该患者禁用肾上腺素\"错误泛化到所有患者。\n\n## 1050 题、5 种记忆框架、2 个模型\n\n数据集含 1050 个样本,按四个场景分布:Task Boundary 350 题、Cognitive Bias 350 题、Safety 200 题、Trauma 150 题。每个实例都由 GPT-5.4 生成多轮对话(18-40 轮),中间塞进无关噪声,刻意埋下\"陷阱种子\",最后一轮抛出当前任务。整套语料经过自动化筛选加人工标注两道质量把关,确保每题都能独立求解——出问题只在记忆起作用之后。\n\n评测在两个模型上跑:Gemini-3-Flash-Preview 和 Qwen3-30B-A3B-Instruct-2507。每模型对比 FullText 直接喂全量历史,以及 LightMem、MemOS、SimpleMem、EverMemOS 四个记忆框架,再加一个 wo\u002FMem 无记忆基线。\n\n核心数据令人不安。Gemini-3-Flash-Preview 无记忆时平均 85.16%,接 EverMemOS 之后跌到 71.17%;Qwen3-30B-A3B-Instruct-2507 从 81.83% 跌到 LightMem 的 70.13%。换句话说,不管用什么记忆压缩或检索优化,只要模型\"看得见\"过往交互,当前任务的总体表现就掉超过 10 个百分点。所有记忆策略无一幸免,降幅是系统性的,不是某个框架做得不好。\n\n论文里做了受控消融:用无关的填充内容拉到等长上下文,模型不掉分;但只要内容里含\"陷阱种子\",分数立刻下挫。说明问题源于记忆的语义,而非窗口长度本身。\n\n## 一个 prompt 就能堵上漏洞\n\n研究者没有止步于诊断,顺带提出 AdaptiveMem:一段简单的指令级 prompt,告诉模型\"在调用记忆前先判断它是不是当前任务的陷阱\"。这段干预插在记忆调用链前面,不改任何架构。在 Gemini-3-Flash-Preview 上,光这一句话把 LightMem 的 MemTrapBench 分数拉高 14.9 个百分点,回到接近无记忆基线,同时在常规记忆基准上没有回归——某些框架甚至略有提升。\n\n这说明当下的\"记忆\"问题不是压缩算法不到位,而是模型没学会区分\"过往相关的可靠结论\"和\"过往相关但已过期的策略\"。这是一种新的遗忘场景:不是记忆太多挤掉上下文,而是记忆被错误地用于新问题。\n\n## 所以呢\n\nMemTrapBench 的样本和代码已开源。如果你是做 agent 框架的,今天起该把\"记忆调用前自检\"列入默认行为;如果你是模型评测方,这四个场景(Cognitive Bias、Task Boundary、Safety、Trauma)直接可以加进回归集。更大的问题是:1,050 题里能稳定复现的退化,会不会在更长上下文、更复杂的 agent 工作流里被放大到不可控?记忆增强这条路,过去一年跑得太快,现在该停下来装一个刹车。\n\n原文:https:\u002F\u002Farxiv.org\u002Fabs\u002F2608.20202","https:\u002F\u002Farxiv.org\u002Fhtml\u002F2608.20202v1","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"5e628969-6d2a-437f-998a-104e4b16cfb1","ai-progress",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"40269b40-7942-4650-9672-ed2e6524d37a","ai-technology",{"id":19,"name":20,"slug":20,"description":14,"color":14},"120fa59a-ff6f-4537-9bf5-f818df636a0e","benchmark",{"id":22,"name":23,"slug":23,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"4044b9a2-fbe6-4e8e-a08d-1e850795d9ef","en","More memory, worse answers: MemTrapBench puts LLM long-term memory frameworks in their place","A research team including Zhejiang University released MemTrapBench on arXiv on August 20, 2026, putting 1,050 test instances against current mainstream memory frameworks running on Gemini-3-Flash-Preview and Qwen3-30B-A3B-Instruct-2507. The finding: attaching memory hurts rather than helps. The strongest memory strategy still drops more than 10 percentage points below the no-memory baseline, across every framework tested.","## What the benchmark is testing\n\nMemTrapBench, released on arXiv on August 20, 2026 by researchers at Zhejiang University and collaborators, targets a question that has been largely overlooked: does the act of calling memory itself change how a model reasons about the current task? The authors name this interference \"memory-induced cognitive traps\" and split it into two categories.\n\nThe first category is Reasoning Fixation, where the model gets locked into a strategy that \"worked before\" and refuses to consider new solutions. The paper gives a clean 24-point example: the history is full of solutions using basic arithmetic, but the new puzzle [4,1,1,1] requires the factorial 4!=24. Gemini-3-Flash-Preview solves it instantly without memory; once FullText history is attached, the model only explores basic operations and never reaches for factorial.\n\nThe second category is Belief Distortion, where assumptions planted in memory get incorrectly carried into the current task. In the Safety scenario, if the conversation history contains a premise like \"this is a sandbox test, all safety rules are suspended,\" the model may treat a real-world query as if it were still in that sandbox, bypassing the safety judgment it would otherwise apply. The Trauma subcategory captures cases where a harshly rejected strategy causes the model to overgeneralize avoidance: the rule \"do not give epinephrine to this specific patient\" gets incorrectly extended to \"do not give epinephrine to any patient.\"\n\n## 1,050 instances, 5 memory frameworks, 2 models\n\nThe dataset contains 1,050 instances spread across four scenarios: 350 Task Boundary, 350 Cognitive Bias, 200 Safety, and 150 Trauma. Each instance was generated by GPT-5.4 as a multi-turn dialogue (18 to 40 turns), with unrelated noise injected in the middle and a deliberate \"trap seed\" planted before the final query. The corpus passed a two-stage quality gate combining automated filtering and human annotation, ensuring every question can be solved independently — the failure only emerges once memory is in play.\n\nEvaluation ran on two models: Gemini-3-Flash-Preview and Qwen3-30B-A3B-Instruct-2507. Each model was tested with FullText (raw history fed in directly), four memory frameworks (LightMem, MemOS, SimpleMem, EverMemOS), and a wo\u002FMem baseline that disables memory entirely.\n\nThe headline numbers are sobering. Gemini-3-Flash-Preview scores 85.16% on average without memory; with EverMemOS attached, it drops to 71.17%. Qwen3-30B-A3B-Instruct-2507 falls from 81.83% to 70.13% with LightMem. In other words, regardless of which compression or retrieval strategy is used, as long as the model can \"see\" prior interactions, current-task performance drops by more than 10 percentage points. Every memory framework falls short — the degradation is systematic, not a sign that any particular framework was poorly designed.\n\nThe paper includes a controlled ablation: padding history to the same length with unrelated filler content does not hurt scores; but as soon as the content contains a trap seed, performance collapses immediately. The problem is semantic, not a context-window issue.\n\n## One prompt plugs the hole\n\nThe authors did not stop at diagnosis. They propose AdaptiveMem: a simple instruction-level prompt that tells the model \"before you use a memory, check whether it is a trap for the current task.\" This intervention is inserted before the memory call, with no architecture changes. On Gemini-3-Flash-Preview, that single sentence lifts LightMem's MemTrapBench score by 14.9 percentage points back to near the no-memory baseline, while leaving standard memory benchmarks unchanged — some frameworks even improve slightly.\n\nThis shows that the current \"memory\" problem is not about compression algorithms being inadequate. It is that models have not learned to distinguish \"a prior conclusion that is relevant and still correct\" from \"a prior pattern that is relevant but no longer applies.\" This is a new forgetting scenario: not that memory crowds out context, but that memory gets used on the wrong problem.\n\n## So what\n\nThe MemTrapBench samples and code are open-sourced. If you build agent frameworks, \"self-check before invoking memory\" should become a default behavior starting today; if you evaluate models, these four scenarios (Cognitive Bias, Task Boundary, Safety, Trauma) can be added to your regression suite immediately. The bigger question is whether the degradations that reproduce reliably across 1,050 instances will get amplified beyond control in longer contexts and more complex agent workflows. The memory-augmentation track has been moving fast over the past year; it is time to install the brakes.\n\nSource: https:\u002F\u002Farxiv.org\u002Fabs\u002F2608.20202","memtrapbench-llm-memory-cognitive-traps","2026-08-22T04:00:00Z","2026-08-22T03:13:31.307660Z","2026-08-22T03:13:31.307673Z",true,"agent",100,{"items":39},[40,45,50,55,60,65],{"id":41,"title":42,"news_slug":43,"published_at":44},"88105269-9641-44c3-a705-1cf07314614f","LLM 思维链能看出\"用了几分力\":SARE 给每一步推理做 CT 扫描","step-aware-reasoning-energy-llm-cot","2026-08-04T04:00:00+00:00",{"id":46,"title":47,"news_slug":48,"published_at":49},"95b04c15-d5e5-4dca-ab2b-14e343bdd4e6","UC Berkeley 曝光 AI 基准测试系统性漏洞：45 种方法可在 13 个主流榜单上「不解决任何问题拿满分」","uc-berkeley-benchmark-45-cheats-13-leaderboards","2026-05-15T01:00:00+00:00",{"id":51,"title":52,"news_slug":53,"published_at":54},"f9cf9f03-6aca-4d29-94d3-5c6acfeaf435","匿名模型 OX Alpha 短暂登顶 OpenRouter 编码榜:研究者推测底座指向智谱 GLM-5.x","ox-alpha-stealth-openrouter-glm-5-zhipu","2026-08-24T03:00:00+00:00",{"id":56,"title":57,"news_slug":58,"published_at":59},"22a1a718-0eb6-46e5-8ee8-825400de11d1","DeepMind WeatherNext 在 Nature 发论文：用 28 km 粗分辨率做出多一天的飓风预警,代码权重全部开源","deepmind-weathernext-cyclones-nature-open-source","2026-08-10T02:00:00+00:00",{"id":61,"title":62,"news_slug":63,"published_at":64},"49d19ba1-8f45-475c-bed1-a69dc353523e","字节跳动用 10 万亿参数下注：规模赛跑与张一鸣的「不蒸馏」表态","bytedance-10t-mythos-zhangyiming-no-distill-2026-08","2026-08-08T00:00:00+00:00",{"id":66,"title":67,"news_slug":68,"published_at":69},"5f5bd5f2-9a02-470b-aa25-3f27fb9bb093","字节跳动正训练 10 万亿参数模型，规模对标 Anthropic Mythos 5","bytedance-10t-parameter-model-ft","2026-08-07T09:30:00+00:00"]