[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-ncp-archpreview-next-concept-prediction":3,"topics-all":38,"news-related-644ac630-d5e5-4ee6-9d63-eccc85d811c4":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"644ac630-d5e5-4ee6-9d63-eccc85d811c4","NCP-ArchPreview：一半 token 追平 OLMo-3 训练损失，概念级预测改写预训练经济学","Intern-NCP 团队 28 位作者发布 NCP-ArchPreview：8.9B 参数潜空间语言模型，5.73T token 上联合训练下一 token 与下一概念预测，仅用 51.3% 训练 token 追平 OLMo-3-7B 最终预训练损失，下游宏平均反超 2.45 分。","过去三年，大模型预训练的主流答案是「更多 token、更多算力」。9 月 9 日提交到 arXiv 的一篇技术报告给出了另一条路线：让模型在预测下一个 token 的同时，学会预测「下一个概念」。这篇 NCP-ArchPreview 在 9 月 11 日冲上 Hugging Face 日榜第一，拿到 127 个 upvote——社区显然被这个方向勾起了兴趣。\n\n## token 之上，再预测概念\n\n论文的核心改动，是给标准 next-token prediction（NTP）加了一个并行的概念级目标 Next Concept Prediction（NCP）。模型先从自身隐藏状态里构建一个乘积量化（product-quantized）的概念词表，把跨越多个 token 的语义单元编码成离散概念；一个专门的 Concept Module 负责预测未来的概念，预测结果再回馈到 token 层面，引导后续生成。两个目标端到端联合训练，生成方式仍是标准的 token 级自回归，不引入另一套推理范式。\n\n## 一半 token，追平同款 loss\n\n规模数字是这篇报告最硬的部分：架构放大到 8.9B 参数，在 Dolma-3 数据集上训练 5.73T token，按论文自述，这是迄今规模最大的潜空间语言模型实证。效果上，只消耗 51.3% 的总训练 token，NCP-ArchPreview 就达到了 OLMo-3-7B 的最终预训练损失；完整预训练之后，下游宏平均反超 OLMo-3-7B 2.45 分，其中 GSM8K 提升达 5.99 分。对照实验单独验证了潜空间架构与 NCP 目标各自的增益；与严格参数对齐的 8.9B 基线相比，它用 85% 的计算量即可逼近后者的训练损失。\n\n## 预训练结束，潜空间还在干活\n\n这套概念空间在预训练之后仍有产出：只更新 1700 万参数的 VQ 模块，就能得到一个轻量的领域适配接口；把概念表示注入 DFlash2 投机解码草稿模型，平均接受长度提升 4.17%，开销几乎可以忽略。换句话说，概念层不只服务于训练目标，还能反哺推理加速与领域迁移。\n\n## 名字里就写着「Preview」\n\n冷静看，模型叫 ArchPreview 不是巧合：对比对象是开放基线 OLMo-3-7B，而非前沿闭源模型；「迄今最大潜空间语言模型」是作者自述，独立复现还需要时间。论文署名 Intern-NCP 团队共 28 位作者（含 Dahua Lin、Zhouhan Lin、Bowen Zhou 等），Hugging Face 页面已有 18 个模型引用这篇论文，社区跟进速度不慢。\n\n算力越贵，目标函数的创新就越值钱。当「再加一倍 token」的边际收益持续走低，让每个 token 承载更难的预测任务，可能是比堆卡更划算的 scaling 方向。论文原文见 [arXiv:2609.10715](https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.10715)，社区讨论见 [Hugging Face 论文页](https:\u002F\u002Fhuggingface.co\u002Fpapers\u002F2609.10715)。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.10715","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"7ac06d8e-b074-4147-abfc-ffaa4c6b8744","ai-efficiency",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"120fa59a-ff6f-4537-9bf5-f818df636a0e","benchmark",{"id":19,"name":20,"slug":20,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":22,"name":23,"slug":23,"description":14,"color":14},"4f214978-cac1-4f39-aa4b-f92a0d0934b7","transformer",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"188f9d0a-de0f-4809-a71c-8db5f99aa719","en","NCP-ArchPreview hits OLMo-3 loss with 51.3% of tokens","8.9B latent-space LM adds next-concept prediction to next-token training: 51.3% of tokens matches OLMo-3-7B final loss, +2.45 macro, GSM8K +5.99.","For three years the default answer in LLM pretraining has been \"more tokens, more compute.\" A technical report submitted to arXiv on Sep 9 proposes a different route: while predicting the next token, the model also learns to predict the \"next concept.\" NCP-ArchPreview climbed to #1 on Hugging Face's daily paper ranking on Sep 11 with 127 upvotes — the community is clearly intrigued.\n\n## Beyond tokens: predicting concepts\n\nThe core change adds a parallel concept-level objective, Next Concept Prediction (NCP), on top of standard next-token prediction (NTP). The model builds a product-quantized concept vocabulary directly from its own hidden states, encoding semantic units that span multiple tokens as discrete concepts; a dedicated Concept Module predicts future concepts, and those predictions are fed back to the token level to guide subsequent generation. The two objectives are trained jointly end-to-end, while generation remains standard token-level autoregression — no separate inference paradigm is introduced.\n\n## Half the tokens, the same loss\n\nThe scale numbers are the hardest part of this report: the architecture is scaled to 8.9B parameters and trained on 5.73T tokens from the Dolma-3 dataset — per the authors' own claim, the largest demonstration of a latent-space language model to date. On results: consuming only 51.3% of the total training tokens, NCP-ArchPreview reaches the final pretraining loss of OLMo-3-7B; after full pretraining it outperforms OLMo-3-7B by 2.45 points on the downstream macro-average, including a 5.99-point gain on GSM8K. Controlled experiments isolate the gains attributable to the latent architecture and to the NCP objective separately; against a strictly parameter-aligned 8.9B baseline, it approaches that baseline's training loss using 85% of the computation.\n\n## The latent space keeps working after pretraining\n\nThe concept space stays useful once pretraining ends: updating just the 17M-parameter VQ module yields a lightweight interface for domain adaptation, and injecting concept representations into a DFlash2 speculative-decoding drafter improves mean accepted length by 4.17% with negligible overhead. In other words, the concept layer doesn't just serve the training objective — it feeds back into inference acceleration and domain transfer.\n\n## \"Preview\" is in the name for a reason\n\nA sober read: the model is called ArchPreview for a reason. The comparison baseline is the open OLMo-3-7B, not frontier closed models; \"largest latent-space language model\" is the authors' own claim, and independent replication will take time. The paper is signed by the 28-author Intern-NCP Team (including Dahua Lin, Zhouhan Lin, and Bowen Zhou), and the Hugging Face page already lists 18 models citing the paper — community follow-up is moving fast.\n\nThe more expensive compute gets, the more objective-function innovation is worth. As the marginal returns of \"double the tokens again\" keep falling, making every token carry a harder prediction task may be a cheaper scaling direction than stacking more GPUs. Full paper: [arXiv:2609.10715](https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.10715); community discussion: [Hugging Face paper page](https:\u002F\u002Fhuggingface.co\u002Fpapers\u002F2609.10715).","ncp-archpreview-next-concept-prediction","2026-09-11T15:10:00Z","2026-09-11T15:08:13.134094Z","2026-09-11T15:08:13.134110Z",true,"agent",95,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"c48681ff-ddbb-402c-ade8-23b584a06aea","更强教师反而教不动学生：Lightning OPD 2.0 剥掉蒸馏中的“文风噪声”","lightning-opd-2-cross-teacher-style-bias","2026-07-30T16:17:15+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"4c4a2a9e-f69b-4985-bd42-97ab2ef4e2ac","Spark-X2.5-4B 开源:4B 跑 1M 上下文,22 项基准打 9B 级 Qwen3.5","spark-x2-5-4b-apache-open-source","2026-09-16T01:30:00+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"0a3f5044-ed20-4c2a-b710-bd26cd276d3e","ALiBi 的隐藏数值故障：长上下文越长，部分注意力头越可能“失明”","alibi-attention-underflow-long-context","2026-08-06T10:30:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"c71b8ee7-9487-4c78-89fd-30bb0368b99e","DeepSeek V4 Flash：284B\u002F13B MoE，成本比 Luna 低 60%","deepseek-v4-flash-0731-intelligence-index-50","2026-08-05T03:00:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"e3f049f5-2f0d-48d2-8e88-246ef006fa16","LoopMTP 给循环 Transformer 装上前瞻路标：固定参数下让每一轮都做不同的事","loopmtp-latent-multi-token-loop-guidance","2026-08-04T13:13:09+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"a151db0c-d832-4df2-ac03-2d4e58b26e99","Kimi K3 跑通 MiniTriton:Moonshot 让 LLM 第一次从零编译出自己的 GPU 编译器","kimi-k3-minitriton-gpu-compiler","2026-07-26T14:00:00+00:00"]