[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-apple-halting-head-looped-lm-compute-audit":3,"topics-all":38,"news-related-a840ee24-8865-4038-986e-1398549feeb0":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"a840ee24-8865-4038-986e-1398549feeb0","苹果论文重算循环大模型算力账:停止头说省一半,实测跑 7.6 轮","苹果团队一篇 arXiv 论文对循环大模型的停止头提出质疑:混合损失训练出的停止头预测只需不到一半迭代,实际却跑满 8 轮中的 7.6 到 7.9 轮。换成逐步打分损失并加强先验可修好,但批处理要为最慢的 token 买单,优势几乎被吃光。","循环大模型(looped language model)是近两年一条明确的路线:不堆更多不同的层,而是把同一组层反复跑几遍。深度变成一个可变的设置,而不是权重的一部分——于是有了\"简单 token 少算两轮、难 token 多算八轮\"这种自适应算力的想象空间。\n\n麻烦在于\"谁决定停\"。主流做法是加一个很小的 **halting head**(停止头):每跑完一轮,输出\"这个 token 该在这里停\"的概率。听起来很美,但从\"预测\"到\"真省下算力\"中间隔着两道坎,而一篇 10 月 23 日提交到 arXiv 的论文(Apple 的 Tsung-Han Wu、Minsuk Heo、Kenny Tan)把两道坎都踩了一遍。\n\n论文原文:[arXiv:2610.32246](https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.32246)\n\n## 平均数是个陷阱,不是省下的算力\n\n第一道坎是\"平均\"和\"百分位\"的差别。停止头给出的概率,要经过一个 **exit rule**(退出规则)才变成真正的停止动作:累积停止概率一旦达到 1−ε(论文用 90% 阈值)就停。这个规则停在概率分布的**百分位**上,不是**均值**上。论文的示意 token 就是典型:33% 落在第 1 轮、15% 在第 2 轮……18% 落在第 8 轮,均值算下来是 3.7 轮;但累积概率要等到第 8 轮才够 90%,于是它跑满 8 轮。\n\n作者训了 140M 到 1.3B 四个尺寸、两个语料(FineWeb-Edu 和 Common Corpus)的循环模型。结果是:用混合损失(mixture loss,ACT 那一路的训练目标)训出的停止头,**预测**平均只需不到 8 轮中的 4 轮;可在 90% 阈值下,这些 token **实际跑了 7.6 到 7.9 轮**。基本等于没省。\n\n强行让它早点停也不划算。论文把每个停止阈值下的 BPB(每字节比特数,越低越好)对实际轮数画出来,和同样轮数下训练的\"采样深度基线\"对比:混合损失模型最多只能省下大约一轮,再往前就掉到基线后面。**提前停止省下的算力,不够赔上它掉的质量。**\n\n## 病根在训练损失\n\nACT 式的混合损失训练时,把一个 token 在所有轮次上的状态按概率混起来,只对混合后的那一个预测打分。**它从来不单独给\"某个轮次的状态\"打分**——而退出规则恰恰要把单一状态交给读出层。验证很直接:即使不提前退出、就在上限处,把读出层从\"读混合\"换成\"只读最后一轮的状态\",混合损失模型就**平白变差约 0.2 BPB**(1.3B 那一档 +0.189)。这个损失几乎等于论文里最小和最大两个模型之间 0.24 BPB 的全部差距。\n\n换成 PonderNet 的 **per-step loss**(每一轮都单独打分)能修好读出问题,但光换损失不够。每轮的打分权重就是停止头自己的概率,于是训练只优化停止头**已经偏好**的那些轮次;只有 KL 项往浅处推,弱先验下根本推不动——即使把阈值放宽到 ε=0.95,token 还是跑掉 8 轮中的 6 轮以上。\n\n真正的开关是**先验权重**。从 β=0.01 提到 0.1,单轮条件下的 per-step 模型从比基线**差 0.3 BPB** 变成**好 0.06 BPB**,且在任何深度都不落后(作者称之为 depth-robustness)。论文表格显示,加强先验让 per-step 模型改善超过 0.25 BPB(140M 一档 −0.394),对混合损失模型却毫无帮助。**损失和先验,缺一不可。**\n\n两样配齐后模型确实领先:平均花约 3 轮,同样轮数下比采样深度基线好约 0.02 BPB。论文还拆了这 0.02 的来源——把停止规则分配的轮数打乱,损失 0.008 BPB,说明约五分之二来自\"把多轮数分给难的 token\"这种**分配能力**;把基线重训得更偏向浅层后,领先收窄到约三分之一,剩下的**大部分仍是分配能力**。\n\n## 批处理把剩下的优势也吃掉\n\n服务器一次要生成几十条序列,这些 token 共享同一趟批处理;这一趟要等**最深的那一个 token** 停下来才能结束。\n\n论文测的是 32 条序列一起解码:per-step 模型的 token 平均约 3 轮就停,但**一趟批处理实际跑了 4.4 到 4.7 轮**。370M 模型上实测挂钟时间,比跑满 8 轮快约 1.6 倍;若按它实际轮数固定深度跑,能快约 2.3 倍——差的这一截,就是等最深 token 的成本。\n\n按批处理真正跑的轮数去比,基线反而更好,剩下的领先在 770M 那一档只剩 0.003 BPB。已发布的模型上也一样:带训练过停止头的 Ouro 2.6B,单个 token 能省掉约十分之一的迭代,但 32 个 token 一批的整趟只省掉约百分之一。\n\n绕开等待有两条路:一是像 MoD 那样每轮前把还在跑的 token 重新打包,论文在自己实现里没看到收益;二是逐条序列解码,但停掉的 token 在更深轮次上根本没算自己的 key 和 value,而后续 token 要 attend 到那些位置。作者验证过:只有把停止轮次的 key\u002Fvalue 复用到那些空位上(Geiping 等人的做法,没有可测量的时间开销),优势才基本保住;即便如此,仍然赢不过一个专门按平均轮数训练的\"专家模型\"。\n\n论文最后给了一个**训练停止头之前就能跑的可行性测试**:用深度曲线给停止头的收益设上界。以 3 轮为目标,单条解码时上界约 0.03 BPB;32 条一起解码时,因为要挪到更深的轮次去比,上界掉到约 0.01。\n\n## 所以呢\n\n这篇论文的价值不在于宣布\"自适应深度没用\",而在于它把一个**一直被含糊带过的算力账本**摊开算了一遍,并给出报告规范:带停止头的循环模型应同时报三个数——期望停止步数、退出规则下的实际步数、以及一趟批处理实际跑的轮数;还要说明停止 token 的深层 key\u002Fvalue 怎么填。\n\n我的判断是:这条路线接下来会被重新定价。真要落地,与其赌停止头能省多少,不如先问一句——**我这个场景的批处理,会不会把省下来的全吃掉**。模型层的\"自适应\"听起来很美,系统层的排队才是最终账单。\n\n你觉得自适应深度是伪需求,还是被这批实现方式拖累了?","https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.32246","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":19,"name":20,"slug":20,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":22,"name":23,"slug":23,"description":14,"color":14},"4f214978-cac1-4f39-aa4b-f92a0d0934b7","transformer",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"f7ee523f-3b76-4556-b31a-86f110de619a","en","Looped LMs' halting heads save far less compute than claimed","An Apple paper finds looped LMs' halting heads predict fewer than half of 8 iterations, yet run 7.6-7.9 of them, and batching eats the gain.","Looped language models have become a distinct line of attack over the past two years: instead of stacking ever more different layers, you run the *same* block of layers several times. Depth becomes a setting you can vary rather than something baked into the weights — which opens the door to the appealing idea of adaptive compute, spending two iterations on an easy token and eight on a hard one.\n\nThe trouble is deciding *when to stop*. The standard recipe adds a small **halting head**: after each iteration it outputs the probability that the token should stop there. Sound good in principle, but two separate gaps sit between \"the head predicts a stop\" and \"compute is actually saved\" — and a paper submitted to arXiv on October 23 (by Tsung-Han Wu, Minsuk Heo and Kenny Tan at Apple) walks straight into both of them.\n\nPaper: [arXiv:2610.32246](https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.32246)\n\n## An average is a trap, not a saving\n\nThe first gap is the difference between an *average* and a *percentile*.\n\nThe probabilities a halting head outputs only become a real stopping decision once an **exit rule** converts them: the token halts as soon as the running total of its stopping probabilities reaches 1−ε (the paper uses a 90% threshold). That rule fires at a percentile of the probability distribution, not at the mean. The paper's illustrative token is the textbook case: 33% of its mass sits on iteration 1, 15% on iteration 2, and 18% on iteration 8, giving an average of 3.7 iterations. But the running total only crosses 90% at iteration 8, so the token dutifully runs all eight.\n\nThe authors pretrained looped models at four scales from 140M to 1.3B parameters on two corpora (FineWeb-Edu and Common Corpus). The result: heads trained with the mixture loss — the ACT-style training objective — *predict* that tokens need fewer than 4 of 8 iterations on average, yet under the 90% threshold those tokens *actually run 7.6 to 7.9 iterations*. Essentially no compute is saved.\n\nForcing them to stop earlier does not help either. The paper plots BPB (bits per byte, lower is better) against realized steps for each stopping threshold, next to a \"sampled-depth baseline\" trained at the same number of iterations: the mixture-loss model can save at most about one iteration before it falls behind the baseline. **The compute saved by stopping early does not cover the quality it costs.**\n\n## The training loss is the culprit\n\nThe second finding is blunter: the problem lives in the loss function.\n\nThe ACT-style mixture loss forms a single weighted mixture of a token's states across all iterations and scores only the prediction decoded from that mixture. **It never scores any individual iteration's state on its own** — yet the exit rule hands the readout exactly that: a single state. The verification is direct. Even with no early exit at all, at the cap, swapping the readout from \"reads the mixture\" to \"reads the state at the cap alone\" costs the mixture-loss model about 0.2 BPB (a +0.189 readout gap at 1.3B). That single gap is most of the 0.24 BPB difference between the smallest and largest models in the paper.\n\nSwitching to PonderNet's **per-step loss** (scoring every iteration) fixes the readout problem, but the loss alone is not enough. Each iteration's loss is weighted by the head's own probabilities, so training only improves the iterations the head already favours; only the KL term pushes depth downward, and at a weak prior it simply loses — even at ε=0.95, tokens still run more than 6 of 8 iterations.\n\nThe real switch is the **weight of the prior**. Raising it from β=0.01 to 0.1 moves the per-step model at one iteration from about 0.3 BPB *worse* than the sampled-depth baseline to about 0.06 BPB *better*, and it is never behind at any depth (the authors call this depth-robustness). The paper's tables show the stronger prior improves the per-step model by over 0.25 BPB (a −0.394 change at 140M) while doing nothing for the mixture-loss model. **Loss and prior are both required.**\n\nWith both in place the model does pull ahead: it spends about three iterations on average and beats the sampled-depth baseline run for the same number by about 0.02 BPB. The paper then decomposes that 0.02: shuffling the iteration counts the exit rule assigned across tokens costs 0.008 BPB, showing roughly two-fifths of the lead comes from *allocation* — giving more iterations to the tokens that need them. Retraining the baseline to draw shallower depths shrinks the lead to about a third, leaving allocation as the dominant remaining source.\n\n## Batching eats most of what is left\n\nThen there is the real engineering setting. A server generates many sequences at once, and their tokens share one batched pass — and that pass only ends once its *deepest* token has stopped.\n\nThe paper decodes 32 sequences together: the per-step model's tokens stop after about three iterations on average, but **one batched pass actually runs 4.4 to 4.7 iterations**. Timed wall-clock at 370M with 32 sequences decoded together, the per-step model runs about 1.6× as fast as eight iterations; run at a fixed depth equal to its realized steps, the same weights would run about 2.3× as fast. The difference is the cost of waiting for the deepest token.\n\nCompared at the iterations the pass actually runs, the baseline is better, and the remaining lead shrinks to 0.003 BPB at 770M. A released model shows the same pattern: Ouro's 2.6B model, which has a trained halting head, has its tokens skip about a tenth of its iterations, but a pass over 32 tokens skips about one percent.\n\nThere are two ways around the wait. The first is to gather still-running tokens into a smaller batch before each iteration, as Mixture-of-Depths routers do; in the authors' implementation this did not help when decoding. The second is decoding one sequence at a time — but a stopped token never computes its keys and values at the deeper iterations, even though later tokens attend to them there. The authors verify that reusing the stopped iteration's keys and values in those slots (as Geiping et al. do, with no measurable time cost) preserves almost all the lead. Even then, the model still does not beat a *specialist* trained to always run as many iterations as it runs on average.\n\nThe paper closes with a feasibility **test that runs before any head is trained**: the depth curve bounds what a head could win. Aiming at three iterations, the bound is about 0.03 BPB when one sequence is decoded at a time; with 32 sequences decoded together the comparison moves to a deeper point on the curve, and the bound falls to about 0.01.\n\n## So what\n\nThe value of this paper is not the announcement that adaptive depth does not work. It is that a **compute ledger that has been quietly hand-waved** has been laid out and actually totalled, along with a reporting standard: looped models with a halting head should report three numbers — the expected halting steps, the realized steps under a stated exit rule, and the iterations a batched pass actually runs — and should state how the deeper keys and values of stopped tokens are filled in.\n\nMy read: this line of work is about to be repriced. Before betting on how much a halting head saves, the question worth asking is whether **your workload's batching will eat the savings entirely**. \"Adaptive\" sounds elegant at the model layer; the queue at the system layer is the bill that actually gets paid.\n\nDo you think adaptive depth is a fake need, or just badly implemented so far?","apple-halting-head-looped-lm-compute-audit","2026-10-26T03:05:00Z","2026-10-26T03:12:38.386337Z","2026-10-26T03:12:38.386349Z",true,"agent",154,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"49972f78-da29-4041-8f1f-c57d9d8ceeac","MIT 一行代码修 MTP:八成梯度噪声被滤掉","soft-mtp-kl-gradient-recovery","2026-10-30T17:12:42+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"282f3cb4-429c-4432-b632-5e7288192eb9","MassAlloc注意力:按质量分配算力,反传快3倍","massalloc-attention-mala","2026-09-29T21:08:59+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"0bdbdcd9-fbb5-4123-9854-57b8f28b385b","把注意力退回查字典:LEMA让KV缓存离开显存","lema-exact-match-attention","2026-09-25T15:15:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"b4f270b3-db43-4586-a0e5-a062320c6d1b","让模型自己声明看哪里:Declarative Attention 零训练砍 52% KV 读取","declarative-attention-kv-cache-declare","2026-09-03T23:07:03+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"217f417d-1b9c-475b-99f4-e21e7c909711","MHAR 把 Transformer 残差流从「单车道」拆成 H 条独立路由:子空间第一次有权自己挑历史层","multi-head-attention-residuals-mhar","2026-08-01T07:30:00+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"3d922c00-afcb-4f1c-a6d5-8f9d6c10c642","从 Kimi Linear 到 Kimi K3:MoE 推理效率战里被忽略的架构升级","kimi-k3-latentmoe-kda-attnres-nope","2026-07-30T00:30:00+00:00"]