[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-memt-1-0-memory-native-foundation-model":3,"topics-all":38,"news-related-b369ce1a-f8a3-4aba-ba78-c73c097a413d":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"b369ce1a-f8a3-4aba-ba78-c73c097a413d","MemT-1.0：记忆变成模型内置能力，128K 历史存储降 66 倍","10 月 19 日提交 arXiv 的论文主张,记忆不该是模型的附加功能,而应内生于模型计算本身,标题写成\"迈向记忆原生的演进\"。在 128K token 历史长度下,MemT-1.0-Nano 取得约 1.86 倍推理加速,相比全上下文 KV cache 实现 66 倍存储缩减,并放出两档权重。","过去两年大模型的\"记忆\"基本都建在模型之外：RAG 把历史塞进向量库，Agent 框架每轮把检索结果拼回上下文，长期记忆系统定期往提示词追加历史片段。共同前提是模型本体会遗忘，记忆必须靠外挂管道喂进去。历史越长，要么上下文窗口膨胀，要么反复检索、反复付费。\n\n10 月 19 日提交到 arXiv 的一篇论文想换一条路，主张记忆不该是模型的附加功能，而应是模型计算本身内生的能力。\n\n## 记忆块挂在注意力旁边\n\nMemT-1.0 在基座注意力之外并联了一个\"原生记忆块\"：局部记忆块维护多通道记忆矩阵，随交互轮次演化；超参记忆块含可训练的记忆查询、键、值投影，以及擦除、写入、遗忘三个门控和输出门。\n\n推理时，记忆注意力读取当前记忆状态，归一化加门控后叠加到基座注意力输出上；每轮对话或片段结束时，由 GDN-2 执行 token 级记忆更新。模型不必重读原始上下文，历史已被压缩进一个持续演化的状态。承接在 Metis 基础上，它用 Gated DeltaNet-2 实现该更新，并引入 Mixture-of-Storage 扩展记忆容量。\n\n## 两档权重：Nano 挂 35B 底座，Flash 挂 DeepSeek\n\n- **MemT-1.0-Nano**：基座 Qwen3.6-35B-A3B，总参 36B、激活 3B，可训练记忆参数 451M，动态记忆状态 10M。\n- **MemT-1.0-Flash**：基座 DeepSeek-V4-Flash，总参 288B、激活 17B，可训练记忆参数 3.61B，动态记忆状态 336M。\n\nFlash 的 336M 记忆状态相对 288B 的模型体量几乎可以忽略。这对应摘要里的关键数字：128K token 历史长度下，Nano 取得约 1.86 倍端到端推理加速，相比全上下文 KV cache 实现 66 倍存储缩减。机制上说得通——状态不随历史长度线性增长，加速和存储收益都由这个恒定性推出。\n\n## 不是全面领先\n\nREADME 的评测小结很克制：在 11 个记忆基准上，Flash 与 Nano 分别在**被评测的参数化记忆方法**中排名第一和第二；但强文本记忆系统整体仍领先，尤其在需要检索细节证据的超长历史上。\n\n这句话的信息量比第一名更大。\"参数化记忆方法\"是被限定过的比较范围——真正的 RAG 和长上下文方案没有被拉进同一张榜单，而恰恰是后者在超长历史的细节检索上占优。准确读法是：在\"把记忆写进权重和状态\"这条路线内部它做到了最好，整体版图上还没赢。\n\n## 真正的赌注\n\n记忆内生化听起来像纯工程收益，但它押的是个关于模型性质的重构判断：持久状态是固定大小的常量，历史必须被压缩进一个不随时间膨胀的容器，这与\"上下文越长越好\"的主流假设相反。1.86 倍加速和 66 倍存储正是该假设成立时的红利。\n\n代价也写在同一份材料里。如果任务是在十万 token 里精确捞出一行证据，把记忆内化未必比做检索划算。真正吃到红利的是另一类场景：交互轮次极长、经验高度重复、不需要逐字回溯原始记录的 Agent 任务。论文里的经验引导式 Agent 编码正是这个方向——经验存进原生记忆，执行时通过记忆注意力读取，不占上下文。\n\n论文与代码已公开：arXiv:2610.26825，代码 github.com\u002FMemTensor\u002FMemT-1.0，权重在 Hugging Face 的 MemTensor 集合下。作者名单里有 Jiajun Shen（DeepSeek 创始人）。\n\n## 所以呢\n\n\"给模型接一个记忆库\"和\"让模型自己长出记忆\"是两种技术观：前者假设记忆是可外挂的工程问题；后者假设记忆是模型能力的一部分。后者目前只在特定形状的任务上跑赢同行——但一旦成立，Agent 的成本结构会被改写，而不只是被优化。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.26825","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"5e628969-6d2a-437f-998a-104e4b16cfb1","ai-progress",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",{"id":19,"name":20,"slug":20,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":22,"name":23,"slug":23,"description":14,"color":14},"4f214978-cac1-4f39-aa4b-f92a0d0934b7","transformer",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"fec7e5b8-3fe4-40d9-b452-bdf663c46c68","en","MemT-1.0 Turns Memory Into an Intrinsic Model Capability","Memory becomes intrinsic to model computation, not an external pipeline: 1.86x speedup and 66x less storage than KV cache at 128K history.","For most of the past two years, memory in large language models has lived outside the model. RAG pushes history into a vector store, agent frameworks splice retrieved chunks back into the context every turn, and long-term memory systems periodically append historical fragments to the prompt. All of them share one premise: the model itself forgets, so memory has to be continuously fed in through an external pipeline. That premise carries a cost, because longer history means either a growing context window or repeated retrieval and repeated billing.\n\nMemT-1.0 takes a different route. Submitted to arXiv on October 19, it argues that memory should not be an add-on feature but an intrinsic part of model computation, and the paper title states that goal directly: \"Towards Memory-Native Evolution.\"\n\n## The Memory Block Sits Beside Attention\n\nThe architecture is not mysterious, but the placement is deliberate. MemT-1.0 runs a \"Native Memory Block\" in parallel with the backbone model's attention, and that block splits into two parts. The Local Memory Block maintains a dynamic, multi-channel memory matrix that evolves across interaction steps. The Hyper Memory Block holds trainable memory query, key, and value projections, along with erase, write, and forget gates plus an output gate.\n\nAt inference time the data flow is straightforward: memory attention reads the current memory state, normalizes and gates the readout, and adds it to the backbone attention output. At the end of each turn or segment, GDN-2 performs token-level memory updates for subsequent steps. The model never has to go back and reread the original context, because history has already been compressed into a continuously evolving state.\n\nBuilding on Metis, MemT-1.0 uses Gated DeltaNet-2 for that token-level update and introduces Mixture-of-Storage (MoS) to scale native memory capacity.\n\n## Two Checkpoints: Nano on a 35B Backbone, Flash on DeepSeek\n\nThe official repository ships a model family with a wide gap between the two specifications:\n\n- **MemT-1.0-Nano**: built on Qwen3.6-35B-A3B, 36B total parameters, 3B activated, 451M trainable memory parameters, and a 10M dynamic memory state.\n- **MemT-1.0-Flash**: built on DeepSeek-V4-Flash, 288B total parameters, 17B activated, 3.61B trainable memory parameters, and a 336M dynamic memory state.\n\nThe comparison between memory state and total parameters is the part worth watching. Flash carries a 336M memory state against a 288B model body, which is negligible in relative terms. That maps directly onto the headline number in the paper abstract: at a history length of 128K tokens, MemT-1.0-Nano achieves approximately 1.86x end-to-end inference speedup and a 66x reduction in persistent storage relative to full-context KV cache. The repository phrases it as the persistent state remaining constant in size, and the mechanism is self-consistent: because the state does not grow linearly with history length, both the speedup and the storage reduction follow from that constancy.\n\nTraining runs in two stages, continued pre-training followed by post-training, supported by 34B tokens of data. For deployment, the Nano version serves through SGLang and the official example runs on two GPUs, while Flash has its own launch script.\n\n## The Honest Caveat: Not a Sweeping Win\n\nNeither the abstract nor the repository dodges this point. The README's evaluation summary is notably restrained. Across 11 memory benchmarks, MemT-1.0-Flash and MemT-1.0-Nano rank first and second respectively **among the evaluated parametric-memory methods**. But strong textual-memory systems remain ahead overall, particularly on long histories that require detailed evidence retrieval.\n\nThat sentence carries more information than the ranking itself. \"Parametric-memory methods\" is a bounded comparison set: genuine RAG and long-context solutions were not pulled onto the same leaderboard, and it is precisely the latter that holds the advantage on detail retrieval over very long histories. So the accurate reading is that MemT-1.0 did best on the route of writing memory into weights and state. It has not won the wider memory landscape.\n\n## What This Route Actually Bets On\n\nMemory internalization sounds like a pure engineering gain, but what it really wagers is a structural claim about what a model is. A persistent state that stays a fixed-size constant means history must be compressed into a container that does not inflate over time, which runs against the mainstream assumption that longer context is better context. The 1.86x speedup and 66x storage reduction are the dividends of that assumption holding.\n\nThe cost sits in the same document, because strong textual-memory systems still lead on detail retrieval over very long histories. In other words, if your task is fundamentally about pulling one exact line out of a hundred thousand tokens, internalizing memory into the model may not beat doing retrieval honestly. The scenarios that actually collect the dividend look different: very long interaction histories, highly repetitive experience, and tasks that never need to trace back to the original record verbatim. The paper's experience-guided agentic coding points in exactly that direction, storing retrieved experience in native memory and reading it through memory attention during execution, without spending task context on it.\n\nThe gap between the two specifications also makes sense in that light: a lightweight version validating that the mechanism works, and a heavy version testing the capacity ceiling.\n\nPaper and code are both public, at arXiv:2610.26825, in the repository github.com\u002FMemTensor\u002FMemT-1.0, with weights under the MemTensor collection on Hugging Face. Worth noting alongside that: the author list includes Jiajun Shen, founder of DeepSeek, along with a long roster of names including Jiawei Liu and Tao Jiang, on a team of more than seventy.\n\n## So What\n\nAttaching a memory store to a model and letting a model grow its own memory are two different technical philosophies. The first assumes the model is stable and that memory is an outsourceable engineering problem. The second assumes memory is part of model capability, and that it should be trained and evolved alongside the parameters. The second currently wins only on tasks with a particular shape. But if it holds, it does not merely optimize the cost structure of agents. It rewrites it.","memt-1-0-memory-native-foundation-model","2026-10-20T11:07:23Z","2026-10-20T11:07:25.287883Z","2026-10-20T11:07:25.287897Z",true,"agent",203,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"2266cea6-06f1-4932-8905-1bc3f2e5a8c0","Meta FAIR 字节蒸馏研究:End-Of-Token 渐近反超 token 蒸馏 4%,数据只需 1\u002F6","meta-fair-byte-distillation-token-ceiling-2026-09","2026-09-15T02:00:00+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"3d922c00-afcb-4f1c-a6d5-8f9d6c10c642","从 Kimi Linear 到 Kimi K3:MoE 推理效率战里被忽略的架构升级","kimi-k3-latentmoe-kda-attnres-nope","2026-07-30T00:30:00+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"b5638cab-a4d6-44ac-9230-32ed0a4cba9d","ARMT 把「记忆」焊进 Transformer:用恒定显存换无限上下文","armt-associative-recurrent-memory-transformer","2026-07-23T00:00:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"d2844cbb-b70e-469c-95b8-71cee8d735a6","给 Transformer 装上「CNN 鼻子」:用 0.01% 的参数量换 benchmark 普涨","transformer-cnn-nose-0-01-percent","2026-07-22T12:10:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"067a3f68-9bc8-4486-9715-5a391e537909","ETH 统一评测循环线性注意力：Kimi Delta Attention 损失最低","eth-zurich-clvr-kda","2026-07-12T04:00:00+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"29774f38-c361-4dca-b11a-c14df2fc84d9","HiLS 把\"无限上下文\"从口号变成数学:让稀疏注意力首次跑赢 Full Attention","hils-hierarchical-landmark-sparse","2026-07-07T14:00:00+00:00"]