[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-deep-fusion-memory-parametric-llm":3,"topics-all":35,"news-related-3386316f-bb1f-4310-952c-8e2f4b6b8f30":54},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":21,"news_slug":28,"published_at":29,"created_at":30,"modified_at":31,"is_published":32,"publish_type":33,"image_url":14,"view_count":34},"3386316f-bb1f-4310-952c-8e2f4b6b8f30","DFM 深度融合记忆:免检索注入知识,领先 12.71%","Deep Fusion Memory 把外部语料炼成按块组织的参数记忆:主干冻结、推理零检索,构建训练数据比 token 级方案少 2048 倍检索条目。六项 QA 上,它较最强基线在 Mistral-7B-v0.3 提升 12.71%、Qwen3-8B-Base 提升 11.19%。","给大模型补外部知识,长期只有两道主菜:继续预训练,有灾难性遗忘的风险;推理时挂 RAG,要付出可观的在线检索开销。Deep Fusion Memory(DFM)选了第三条路——把语料「炼」成参数记忆,推理时一次检索都不用做。这篇论文(arXiv:2610.25100)10 月 18 日提交,由 Rubin Wei 等八位作者完成,目前已在 Hugging Face 论文日榜收获 49 个赞。\n\n## 参数记忆的两个老毛病,一次治两个\n\n外部参数记忆这条路早已有之,但论文点名它的两大痛点:一是训练数据要靠大量离线检索来构造,成本高;二是多数方案只在输出层做插值,融合太浅,知识并没有真正进入模型的计算过程。DFM 的解法是把记忆做粗、做深——按「块」组织记忆,并让它融入中间隐状态,两个痛点各对应一刀。\n\n## 机制:预测器加门控,主干全程冻结\n\nDFM 的核心是两个部件。其一,因果记忆预测器:从当前上文直接预测出该用的记忆;训练时用语料中检索到的块嵌入当监督信号,推理时预测器自己生成,不再查库。其二,门控交叉注意力:预测出的记忆在块内所有 token 之间共享,融进中间隐状态,而不是在输出层贴补丁;块级粒度也把构库开销从 token 级大幅拉低。整个记忆预测与融合端到端训练,语言模型主干全程冻结,推理阶段不需要任何显式检索。换句话说,检索这件苦差事被挪到了训练阶段,由块嵌入监督「教会」预测器,推理时模型带着炼好的记忆直接干活。\n\n## 账面:构库成本与效果\n\n效率账很直接:与构造 token 级记忆目标的既有参数记忆方案相比,DFM 构建训练数据少用 32 倍检索查询、2048 倍检索条目。效果账也齐:六项 QA 任务上,较最强基线在 Mistral-7B-v0.3 上提升 12.71%,在 Qwen3-8B-Base 上提升 11.19%;领域适配实验里,以 Qwen3-4B-Base 为底座平均涨 6.51 分,以 Qwen3-8B-Base 为底座平均涨 4.38 分,大小两档底座都验证过。\n\n## 这条路和这一周的记忆热是什么关系\n\n最近几天这个赛道相当热闹:前几天刚有评测结论说,朴素 RAG 就能打败复杂的智能体记忆系统;也有把记忆做成模型内置能力、或者专做记忆写入的开源模型。DFM 的差异化姿势很清楚:推理零检索、主干零改动——前者把延迟成本从推理路径上拿掉,后者直接绕开继续预训练的灾难性遗忘风险。代价并非消失,而是转移到离线构库;32 倍与 2048 倍的降幅,正是冲着这个瓶颈去的。给做企业知识库或领域适配的人一句「所以呢」:知识注入的工程选项,正在从「推理时挂检索」向「训练时炼记忆」回摆;主干冻结意味着可以按语料批量炼、随时换,而不必重训基座——这条路值不值得跟,接下来就看社区在更大基座上的复现。\n\n参考:[arXiv:2610.25100](https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.25100) · [HF 论文页](https:\u002F\u002Fhuggingface.co\u002Fpapers\u002F2610.25100)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.25100","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18],{"id":12,"name":13,"slug":13,"description":14,"color":14},"5e628969-6d2a-437f-998a-104e4b16cfb1","ai-progress",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",{"id":19,"name":20,"slug":20,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",[22],{"id":23,"lang":24,"title":25,"summary":26,"content":27},"ba538520-41d1-471f-a81f-78eed2d85b4d","en","DFM: Parametric Memory for LLMs, No Retrieval at Inference","DFM bakes a corpus into chunk-level parametric memory: no inference retrieval, frozen backbone, and +12.71% QA over the strongest baseline.","Giving large language models external knowledge has long meant choosing between two costs: continued pretraining, which risks catastrophic forgetting, and inference-time RAG, which adds substantial online retrieval overhead. Deep Fusion Memory (DFM) takes a third path — it distills a corpus into parametric memory, so inference does no retrieval at all. The paper (arXiv:2610.25100, submitted October 18, by Rubin Wei and seven co-authors) currently has 49 upvotes on the Hugging Face daily papers board.\n\n## Two old problems of parametric memory, addressed at once\n\nExternal parametric memory is not new, and the paper names its two chronic pains: building the training data requires extensive offline search, and most approaches only interpolate at the output level — a shallow fusion that never really enters the model's computation. DFM's answer is to make memory coarser and deeper at the same time: organize it by chunk, and fuse it into intermediate hidden states. One cut per pain.\n\n## Mechanism: a predictor plus a gate, with the backbone frozen\n\nDFM has two core parts. First, a causal memory predictor: it generates the needed memory directly from the preceding context. During training, retrieved chunk embeddings from the corpus serve as supervision; at inference, the predictor generates memory on its own — no lookups. Second, gated cross-attention: the predicted memory is shared across all tokens within a chunk and fused into intermediate hidden states rather than patched on at the output layer; the chunk-level granularity also slashes construction costs compared with token-level targets. Memory prediction and fusion are trained end to end while the language model backbone stays frozen, and the inference stage requires no explicit retrieval.\n\n## The numbers: construction cost and effectiveness\n\nThe efficiency account is direct: compared with existing parametric-memory approaches that construct token-level memory targets, DFM needs 32x fewer retrieval queries and 2048x fewer retrieved items to build its training data. The effectiveness account is complete as well: across six QA tasks, DFM beats the strongest baseline by 12.71% on Mistral-7B-v0.3 and by 11.19% on Qwen3-8B-Base; in domain adaptation, it gains an average of 6.51 points on Qwen3-4B-Base and 4.38 points on Qwen3-8B-Base — validated on both smaller and larger backbones.\n\n## How this fits this week's memory boom\n\nThe track has been busy lately: days ago an evaluation concluded that naive RAG already beats complex agentic memory systems, and open models have shipped that internalize memory or specialize in memory writing. DFM's position is crisp: zero retrieval at inference, zero changes to the backbone — the first removes latency from the serving path, the second sidesteps the catastrophic-forgetting risk of continued pretraining. The cost does not vanish; it moves to offline construction, and the 32x\u002F2048x reductions aim exactly at that bottleneck. For teams building enterprise knowledge bases or domain adaptations, the takeaway is this: the engineering menu for knowledge injection is swinging back from \"hang retrieval on inference\" toward \"distill memory at training time.\" A frozen backbone means you can batch-distill per corpus and swap memory without retraining the base model — whether the path holds up will show in community replications on larger backbones.\n\nReference: [arXiv:2610.25100](https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.25100) · [HF paper page](https:\u002F\u002Fhuggingface.co\u002Fpapers\u002F2610.25100)","deep-fusion-memory-parametric-llm","2026-10-25T17:20:21Z","2026-10-25T17:20:30.124494Z","2026-10-25T17:20:30.124504Z",true,"agent",245,[36,45],{"slug":37,"tag_slug":37,"title_zh":38,"title_en":39,"intro_zh":40,"intro_en":41,"id":42,"is_active":32,"created_at":43,"modified_at":44},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":46,"tag_slug":46,"title_zh":47,"title_en":48,"intro_zh":49,"intro_en":50,"id":51,"is_active":32,"created_at":52,"modified_at":53},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":55},[56,61,66,71,76,81],{"id":57,"title":58,"news_slug":59,"published_at":60},"e8168e92-d16e-452a-991c-aac74dd23ec0","预训练赢了,微调输了:Muon 的反转","muon-adamw-fine-tuning-gap-reversal","2026-11-04T01:12:00+00:00",{"id":62,"title":63,"news_slug":64,"published_at":65},"a6cc8527-e099-488d-ad02-9eb997fa4b96","On-policy失配拆解:恒定数值差与自放大滞后","llm-rl-training-inference-mismatch-dlc","2026-11-02T15:15:34+00:00",{"id":67,"title":68,"news_slug":69,"published_at":70},"3900dc0c-41ec-4dae-be3f-68b0584f5b64","WISD:削弱自己当教师,自蒸馏推理平均涨5.5分","wisd-weakening-self-distillation-reasoning","2026-10-31T01:07:47+00:00",{"id":72,"title":73,"news_slug":74,"published_at":75},"261b6911-0300-4d6b-86dc-a64475e4aa5a","Delta N-gram:给大模型记忆打可拆卸补丁","delta-ngram-forward-only-memory-writes","2026-10-25T23:50:00+00:00",{"id":77,"title":78,"news_slug":79,"published_at":80},"4c6a1f3f-c848-4259-bc88-8672ae5cba0f","MoE 专家跨层共享：省两成参数，残差变双向","meta-expert-cross-layer-moe-expert-sharing","2026-10-25T11:20:00+00:00",{"id":82,"title":83,"news_slug":84,"published_at":85},"8c579d8d-4114-4d7d-92c1-22f5fab0537b","连续流语言模型跑到 8B：Lafa 单模型自验证解码提速 2.54 倍","large-flow-language-models-lafa-self-speculative-decoding","2026-10-21T03:09:03+00:00"]