[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-compose-cl-long-horizon-memorization":3,"topics-all":38,"news-related-2e27016d-b90e-45c7-825a-41fd1e435c80":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"2e27016d-b90e-45c7-825a-41fd1e435c80","JHU 新研究:组合持续学习机制,百任务记忆留存从 1.2% 提到 34.9%","约翰霍普金斯团队定义「长程记忆化」基准:模型连学 100 个问答任务,朴素顺序微调后平均留存只剩 1.2%。论文组合数据\u002F函数\u002F权重三种锚点与合并 LoRA,把留存率提到 34.9%(28 倍),三个数据集全部进入前三。","给大模型连续灌 100 个任务的知识,等第 100 个学完,前面学的还剩多少?Johns Hopkins 团队给出的基线答案很刺眼:1.2%。这篇 9 月 7 日提交到 arXiv 的论文,以 287 赞登上 Hugging Face 9 月 16 日每日论文榜第一。它研究的是大模型微调的老毛病——灾难性遗忘,而结论相当反直觉:别再纠结选哪种防遗忘机制,把它们组合起来。\n\n## 先把「忘性」量化\n\n论文定义了一个叫「长程记忆化」(long-horizon memorization)的设定:模型通过持续监督微调连续学习 100 个问答任务,不保留旧的训练数据,推理时也不告知任务编号。在这个刻度下,朴素顺序微调学完 100 个任务后,平均最终留存率只剩 1.2%——几乎所有知识都被后续更新冲掉了。\n\n为避免结论只对某类数据成立,团队构造了三个 100 任务数据集:Symbol-QA 用随机六字符键映射四字符值,测纯符号记忆;LLM-QA 由 LLM 生成虚构事实;Real-QA 则从十个公开 QA 来源过滤重混而成,贴近真实知识。\n\n## 组合拳怎么打\n\n论文的核心假设:针对遗忘不同来源的机制,组合起来比单打独斗强。组合沿两个设计维度展开:\n\n- **锚点(anchors)决定「保什么」**:数据锚点用生成式回放,函数锚点用对旧模型的自蒸馏,权重锚点用 SI 或 online EWC;\n- **低秩分配规则决定「存哪里」**:共享 LoRA、合并 LoRA、O-LoRA,以及 OSRM 的顺序化改造。\n\n设计空间组合爆炸,团队先用任务级 successive halving 做漏斗搜索(90→45→23→10),再用 2^4 全因子实验测量单个机制与交互效应。最终胜出的配方 si_sd_replay_merge——三锚点全开加合并 LoRA——在三个数据集上全部进入前三,把平均最终留存率从 1.2% 拉到 34.9%,论文称之为 28 倍改进。其中数据锚点与合并 LoRA 贡献最大,且在三个数据集上均呈超可加交互。\n\n## 34.9% 是解药吗\n\n不是,但这可能是第一次有人把这个赛道的方法论摆得如此系统。两点值得注意:其一,34.9% 意味着约三分之二的知识仍会丢,长程记忆远未解决;其二,工程取舍已经清晰——O-LoRA 与 OSRM 的状态随任务数线性增长,而共享\u002F合并 LoRA 的学习器状态保持常数大小,对需要长期滚动更新的生产模型更友好。实验 backbone 为 Qwen3-4B-Base,论文还评估了 100 个任务之后模型在 GSM8K、MATH、MGSM、MMLU-Redux 上的通用能力。代码与三个数据集均已开源([GitHub](https:\u002F\u002Fgithub.com\u002Fcozheyuanzhangde\u002Fcompose-cl))。\n\n## 所以呢\n\n对持续做领域微调的团队,这篇论文的价值不在 34.9% 这个数字,而在「机制可组合、交互可测量」的方法论:与其在单一防遗忘技巧上反复调参,不如把数据、函数、权重三层保护同时上齐。当模型开始像员工一样「入职后持续培训」,记忆留存率就会从学术指标变成成本指标——每忘掉一个任务,都是一次重新训练的真金白银。\n\n论文原文:[arXiv:2609.06986](https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.06986)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.06986","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"120fa59a-ff6f-4537-9bf5-f818df636a0e","benchmark",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",{"id":19,"name":20,"slug":20,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":22,"name":23,"slug":23,"description":14,"color":14},"b9bd9039-fcdb-41a8-b85b-fc1587def2b9","open-source",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"abdb20cb-174f-422e-98af-f4f2b31d9288","en","Composing Continual Learning Mechanisms: 28x Retention Gain","JHU researchers show combining data, function, and weight anchors with merged LoRA lifts 100-task memory retention from 1.2% to 34.9%, a 28-fold improvement.","Feed a language model 100 tasks in a row, and by the time it finishes the last one, how much of what it learned survives? The baseline answer from a Johns Hopkins team is brutal: 1.2%. The paper, submitted to arXiv on Sep 7 and now #1 on Hugging Face's Sep 16 Daily Papers with 287 upvotes, targets the old nemesis of fine-tuning — catastrophic forgetting — and reaches a counterintuitive verdict: stop agonizing over which anti-forgetting mechanism to pick. Compose them.\n\n## Quantifying Forgetfulness First\n\nThe paper defines \"long-horizon memorization\": a model learns 100 query-answer tasks through continual supervised fine-tuning, without keeping earlier training examples and without task identifiers at inference. Under this ruler, naive sequential fine-tuning ends with an average final retention of just 1.2% — nearly everything gets washed away by subsequent updates.\n\nTo keep conclusions from being data-specific, the team built three 100-task datasets: Symbol-QA maps random six-character keys to four-character values for pure symbolic memory; LLM-QA consists of LLM-generated fictional facts; Real-QA is filtered and remixed from ten public QA sources, closer to real-world knowledge.\n\n## How the Combination Works\n\nThe core hypothesis: mechanisms addressing complementary sources of forgetting work better when composed. Compositions run along two design dimensions:\n\n- **Anchors decide what to preserve**: generative replay for the data anchor, self-distillation against the previous model state for the function anchor, SI or online EWC for the weight anchor.\n- **Low-rank allocation rules decide where updates live**: shared LoRA, merged LoRA, O-LoRA, and a sequential adaptation of OSRM.\n\nThe design space explodes combinatorially, so the team first ran task-level successive halving (a 90→45→23→10 funnel), then a 2^4 factorial experiment to measure individual and interaction effects. The winning recipe, si_sd_replay_merge — all three anchors plus merged LoRA — ranks among the top 3 on all three datasets and lifts average final retention from 1.2% to 34.9%, which the paper calls a 28-fold improvement. The data anchor and merged LoRA contribute the largest gains and interact super-additively on all three datasets.\n\n## Is 34.9% the Cure?\n\nNo — but this may be the first time the methodology of this whole track has been laid out so systematically. Two things stand out. First, 34.9% still means roughly two-thirds of the knowledge is lost; long-horizon memory is far from solved. Second, the engineering trade-offs are now explicit: O-LoRA and OSRM state grows linearly with the number of tasks, while shared\u002Fmerged LoRA keeps learner state constant — friendlier for production models that keep rolling forward. The backbone is Qwen3-4B-Base, and the paper also evaluates general capability on GSM8K, MATH, MGSM, and MMLU-Redux after task 100. Code and all three datasets are open-sourced on [GitHub](https:\u002F\u002Fgithub.com\u002Fcozheyuanzhangde\u002Fcompose-cl).\n\n## So What\n\nFor teams doing continual domain fine-tuning, the value here is not the 34.9% number but the methodology — mechanisms are composable and their interactions are measurable. Rather than endlessly tuning a single anti-forgetting trick, stack protection across data, function, and weight layers. As models start to behave like employees who keep receiving on-the-job training, memory retention turns from an academic metric into a cost metric: every task forgotten is real money spent retraining.\n\nPaper: [arXiv:2609.06986](https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.06986)","compose-cl-long-horizon-memorization","2026-09-16T15:10:00Z","2026-09-16T23:06:51.082907Z","2026-09-16T23:06:51.082926Z",true,"agent",63,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"2731ed1c-17c3-4d85-9174-983cf50743e3","地铁售票机上的 AI 大考:2.6GB 端侧模型 91.32 分超 GPT-5.6,规则基线也拿 84.6","metrollm-bench-transit-kiosk-llm","2026-09-12T23:08:18+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"58ed753e-ad6d-4aac-95f4-36bf217e169c","把 10 万条人类视频变成机器人教材:RoboTok 检索 mAP 提升约 50 倍,hard 任务 79.3% 对 19.5%","robotok-retrieval-benchmark-reread","2026-09-06T21:11:25+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"4c4a2a9e-f69b-4985-bd42-97ab2ef4e2ac","Spark-X2.5-4B 开源:4B 跑 1M 上下文,22 项基准打 9B 级 Qwen3.5","spark-x2-5-4b-apache-open-source","2026-09-16T01:30:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"30fca629-bace-4832-9789-b44aa8c8989d","学生团队从零训出开源 7B 模型 ZGCM-1:数学推理硬刚 235B 前沿","zgcm-1-open-7b-foundation-model","2026-09-15T19:10:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"54b86d93-0fd0-4107-9353-9b79a1446f69","NVIDIA 开源 IMO 金牌完整配方:30\u002F42 分、561B 双专家、算力账本全公开","nvidia-nemotron-imo-gold-open-recipe","2026-09-11T17:13:27+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"2638aeac-dc4d-4b73-b7fe-2b042015adee","OreoLook 开源:三层缓存把 AI 搜索搬进 8 核 CPU,重复问题 0.1 毫秒出答案","oreolook-three-layer-cpu-cache","2026-09-10T23:08:36+00:00"]