[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-par-progress-aligned-recurrence-length-extrapolation":3,"topics-all":38,"news-related-368be0c3-aa88-42ae-a524-d0914f2088f7":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"368be0c3-aa88-42ae-a524-d0914f2088f7","PAR:循环模型多跑几轮就崩,两处训练改动修好外推","蚂蚁集团等机构在 arXiv 提出 PAR(进度对齐递推):第 k 轮预测执行完前 k 步后的状态,并把循环状态缩放回首轮范数。Qwen2.5-0.5B 用最多 32 步训练,在 512 步上达 95.5% 准确率(16 倍长度外推),此前方法 RLT 仅 55.7%、显式思维链 0.8%。","让同一组权重在推理时多跑几轮，是绕过\"参数不够\"最便宜的办法——这就是循环语言模型(looped language model)的全部动机。但 2026 年 10 月 20 日上 arXiv 的一篇论文泼了盆冷水：多跑几轮，并不等于能处理更长的题。\n\n## 循环深度不等于推理能力，翻倍就崩\n\n任务设定很朴素。论文用 swap-S5：五个盒子各放一个物体，给出一串交换操作，最后问每个盒子里装的是什么。在\"单查询\"设定下，作者先测了一组固定深度模型的极限：Qwen2.5 从 0.5B 到 7B，训练长度之外直接崩盘。模型只用最多 32 步的序列训练，测试时塞给 512 步序列(相当于训练长度的 16 倍)，0.5B 到 7B 全线只能拿到约 20% 的准确率——差不多等于乱猜。论文摘要里还记了另一条更极端的曲线：同一骨干上做显式思维链(CoT)，在 swap-S5 上只有 0.8%。[论文原文](https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.27628)\n\n换句话说，问题不在参数量。把模型放大到 7B，仍然追不上自己的训练长度；换成让模型把中间步骤写出来，效果反而更糟。\n\n## 两个病根：监督错位与状态尺度漂移\n\n论文把\"多跑几轮不管用\"拆成了两个可以分离的故障。\n\n第一是**监督错位**。只用最终答案做监督时，模型从来不知道\"第 k 轮该推进到哪一步\"——循环的迭代次数和任务进度之间没有任何绑定关系，多轮叠加只会让状态乱飘。\n\n第二是**尺度漂移**。同一组权重被反复应用数百次，循环状态(recurrent state)的数值尺度会一路漂移，训练时没见过的激活幅度一进来就足以把模型带偏。\n\n这两条恰好对应方法的两处改动，而论文用消融实验把功劳分得很清楚。\n\n## PAR：让每一轮推进恰好一步\n\n方法本身朴素得有点反高潮。第一，第 k 轮的预测，目标是\"执行完前 k 步之后的真实状态\"，而不是最终答案——循环轮次和任务进度被显式对齐了，一轮一步。第二，每轮结束后把循环状态重新缩放回它第一轮时的范数(norm anchoring)，把尺度漂移这个变量直接钉死。\n\n结果是：Qwen2.5-0.5B 用最多 32 步训练，在 512 步上达到 95.5% 的平均全状态准确率，正好是 16 倍长度外推。同一骨干下，此前方法 RLT 的报告值是 55.7%，显式思维链是 0.8%。作者补充的一组数据里，Qwen2.5 的 0.5B、1.5B、3B、7B 以及 Qwen3 的 0.6B 到 4B 都稳定超过固定深度基线，其中 Qwen2.5 四个尺寸的最优训练配置在 16 倍长度处达到了 100% 准确率。这篇论文当天也登上了 Hugging Face 每日论文榜首，获 118 赞。[论文卡片](https:\u002F\u002Fhuggingface.co\u002Fpapers\u002F2610.27628)\n\n## 消融实验：两个部件谁都不能少\n\n消融把\"16 倍\"这个数字拆得很干净：去掉范数锚定，外推倍数从 16 倍掉到 2.2 倍以下，基本等于没做；把\"进度对齐目标\"换回最终的答案监督，外推倍数降到最多 8 倍。循环状态对齐和中间预测监督必须同时上，缺一个都塌。\n\n还有一个容易被忽略的工程细节：模型得知道什么时候停下来。论文用一个学出来的\"完成读出\"(completion readout)来决定何时输出答案，不需要外部给定停止深度。\n\n## 没有中间标签怎么办\n\nswap-S5 天然带中间状态，但真实任务通常只有最终答案。论文给了一个折中方案：前缀自蒸馏(prefix self-distillation)——用较短问题前缀反推，诱导出\"第 k 轮对应第 k 步\"这层对应关系，只靠最终答案就能训。摘要的说法是，当中间标签拿不到时，这个办法可以让增益继续存在。\n\n## 真正值得看的部分\n\n这篇论文的分量不在 95.5% 这个数字，而在于它给出了一条被受控实验反复验证的路线：把中间监督和状态尺度一起训，循环深度才真正变成可用的推理资源。\n\n过去两年\"推理时计算扩展\"这条路有一个隐含假设：只要让模型多想一会儿，它就能处理更长的输入。PAR 用一组对照实验说明这个假设在序列状态跟踪上不成立——不是模型不会想，而是没人告诉它每一轮该想什么，以及想过之后内部状态该长什么样。\n\n有意思的是，这两个缺失恰好都是训练侧的问题，不是架构问题。循环模型没有新增任何参数，论文改的只是监督信号和一层缩放。这对正在堆推理预算的团队是个提醒：把预算花在推理时的重复调用上之前，先确认每一轮迭代是否有明确的进度语义。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.27628","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"7ac06d8e-b074-4147-abfc-ffaa4c6b8744","ai-efficiency",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"5e628969-6d2a-437f-998a-104e4b16cfb1","ai-progress",{"id":19,"name":20,"slug":20,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":22,"name":23,"slug":23,"description":14,"color":14},"4f214978-cac1-4f39-aa4b-f92a0d0934b7","transformer",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"14b596d5-be25-43d0-bab1-2c7b4c0e810f","en","PAR: Loop Models Collapse With More Iterations, Two Training Fixes Restore Extrapolation","A group of institutions led by Ant Group propose PAR (Progress-Aligned Recurrence) on arXiv: the prediction at iteration k targets the state after the first k operations, and recurrent states are rescaled back to their first-iteration norms. A Qwen2.5-0.5B model trained on at most 32 moves reaches 95.5% accuracy at 512 moves (16x length extrapolation), where the prior method RLT manages 55.7% and explicit chain-of-thought just 0.8%.","Running the same set of weights for more iterations at inference time is the cheapest way to work around \"not enough parameters\" — that is the entire motivation behind looped language models. But a paper posted to arXiv on October 20, 2026 pours cold water on it: more iterations do not mean you can handle longer problems.\n\n## Recurrent depth is not reasoning capacity; it collapses at 2x\n\nThe task setup is plain. The paper uses swap-S5: five boxes, one object in each, a sequence of swap operations, and finally a question about what is inside each box. Under the \"single-query\" setting, the authors first probe the limits of fixed-depth models: Qwen2.5 from 0.5B to 7B falls apart well past its training length. Trained on sequences of at most 32 moves and then fed 512-move sequences (16 times the training length), the 0.5B-to-7B range reaches only about 20% accuracy — roughly random guessing. The abstract records an even more extreme curve: explicit chain-of-thought (CoT) on the same backbone manages just 0.8% on swap-S5. [Paper](https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.27628)\n\nIn other words, the problem is not parameter count. Scaling to 7B still cannot keep up with the model's own training length; switching to having the model write out intermediate steps performs worse still.\n\n## Two root causes: supervision misalignment and state scale drift\n\nThe paper splits \"running more iterations does not help\" into two separable failures.\n\nThe first is **supervision misalignment**. When only the final answer supervises training, the model is never told \"how far should iteration k advance\" — nothing binds the number of iterations to task progress, so stacking rounds just sends the state drifting.\n\nThe second is **scale drift**. Applying the same weights hundreds of times pushes the scale of the recurrent state steadily out of distribution, and activation magnitudes unseen during training are enough to knock the model off track.\n\nThese map exactly onto the method's two changes, and the ablations assign credit cleanly.\n\n## PAR: make every iteration advance exactly one step\n\nThe method is almost anticlimactic in its simplicity. First, the prediction at iteration k targets \"the true state after the first k operations\" rather than the final answer — loop count and task progress are explicitly aligned, one loop per step. Second, after each loop the recurrent state is rescaled back to its first-iteration norm (norm anchoring), pinning the scale-drift variable down.\n\nThe result: Qwen2.5-0.5B trained on at most 32 moves reaches 95.5% mean full-state accuracy at 512 moves — exactly 16x length extrapolation. On the same backbone, the previously reported RLT method gets 55.7% and explicit chain-of-thought gets 0.8%. The authors add that Qwen2.5 at 0.5B, 1.5B, 3B, and 7B plus Qwen3 from 0.6B to 4B all steadily beat fixed-depth baselines, with the best Qwen2.5 training runs at four sizes hitting 100% accuracy at 16x the training length. The paper also topped Hugging Face's Daily Papers on the day, with 118 upvotes. [Paper card](https:\u002F\u002Fhuggingface.co\u002Fpapers\u002F2610.27628)\n\n## Ablations: neither component is optional\n\nThe ablations take \"16x\" apart cleanly: removing norm anchoring drops the extrapolation factor from 16x to below 2.2x — essentially nothing; replacing the progress-aligned targets with final-answer supervision caps it at 8x. State alignment and intermediate-prediction supervision have to ship together, and dropping either collapses the result.\n\nOne easily missed engineering detail: the model has to know when to stop. The paper uses a learned \"completion readout\" to decide when to emit the answer, without an externally supplied stopping depth.\n\n## What to do when intermediate labels are unavailable\n\nswap-S5 naturally carries intermediate states, but real tasks usually offer only final answers. The paper offers a compromise: prefix self-distillation — inducing the \"iteration k corresponds to step k\" correspondence from shorter problem prefixes, using only final answers for training. The abstract's claim is that when intermediate labels are unavailable, this keeps the gains available.\n\n## The part actually worth watching\n\nThe weight of this paper is not the 95.5% figure but a repeatedly validated route under controlled experiments: train intermediate supervision and state scale together, and only then does recurrent depth become usable reasoning resource.\n\nThe hidden assumption behind \"inference-time compute scaling\" over the past two years is that giving a model more thinking time lets it handle longer inputs. PAR shows with controlled comparisons that this assumption does not hold on sequential state tracking — not because the model cannot think, but because nobody told it what to think at each round, or what its internal state should look like afterward.\n\nWhat is interesting is that both gaps are training-side problems, not architectural ones. Loop models add no parameters at all; the paper changes only the supervision signal and one rescaling step. For teams currently piling on inference budget, that is a reminder: before spending budget on repeated calls at inference time, confirm each iteration carries an explicit progress semantics.","par-progress-aligned-recurrence-length-extrapolation","2026-10-21T11:10:00Z","2026-10-21T11:04:46.959796Z","2026-10-21T11:04:46.959810Z",true,"agent",36,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"b47914fe-d539-440f-ab02-3b089120051f","MultiHashFormer 用多重哈希签名改写 LLM 词表嵌入：3B 模型跑赢标准 Transformer","multihashformer-hash-embedding-3b","2026-06-29T04:20:00+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"addfda39-d64f-4cb9-809a-7dfadd72d421","ECHO:百度把多Token预测搬进主干最后一层","echo-baidu-multi-token-prediction-self-recursion","2026-10-21T01:15:00+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"b369ce1a-f8a3-4aba-ba78-c73c097a413d","MemT-1.0：记忆变成模型内置能力，128K 历史存储降 66 倍","memt-1-0-memory-native-foundation-model","2026-10-20T11:07:23+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"e40de2f9-9ee4-47f3-b1f1-7bf93a4870a9","一个动词翻转工具调用决策,LLM 内部向量现形","llm-tool-call-decision-vector","2026-10-09T13:10:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"fc533af5-9e8a-43c2-8408-2f931a6aa398","EngramEdit:把事实塞进 LLM 的记忆抽屉","engramedit-decoupled-knowledge-llm","2026-10-09T04:00:00+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"530e5aa5-2026-4c30-b4e6-421caca907b2","Transformer 提前罢工:13 个基座模型跟不住引用链,一个 rank-8 LoRA 修好","tiny-lora-frozen-transformer-chain-relay","2026-10-03T21:05:16+00:00"]