[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-f8ea285b-f717-4f82-aafd-6a096ca6cf46":3},{"id":4,"title":5,"summary":6,"original_url":7,"source_id":8,"tags":9,"published_at":23,"created_at":24,"modified_at":25,"is_published":26,"publish_type":27,"image_url":13,"view_count":28},"f8ea285b-f717-4f82-aafd-6a096ca6cf46","Princeton + UCLA 把 Looped Transformer 的残差缩放修对了:DeepLoop 让循环深度终于稳下来","Looped Transformer 想用一份共享参数、多轮迭代去放大模型深度,一直被一个隐形问题卡住:每次回访都让这条残差分支吃到相同更新,前向-反向耦合关系和 untied Transformer 不一样,DeepNorm 那套按层数取 α=β 的经验法则直接失灵,损失曲线在循环层数一上去就抖动甚至发散。\n\nPrinceton 王梦迪和 UCLA 顾全泉团队(arXiv 2607.13491,2026-07-15)给出第一性原理答案。论文把参数被访问 k 次翻译成一个一阶扰动界,引入 visit-alignment coefficient κ_R:在解相关区域 κ_R 退化,界恢复成 DeepNorm 的 1\u002F4 指数;但在保守对齐区域——也就是工程里更常见的状态——指数必须从 1\u002F4 抬到 1\u002F2,随循环数 N 增长。DeepLoop 的核心:沿用 Post-LN DeepNorm 骨架,只把残差缩放系数改成 α=(2N)^{1\u002F2}、β=(8N)^{-1\u002F2},计算量几乎没变,循环深度第一次能稳住。\n\nGPT-2 small\u002Fmedium 上的实验很直接:不循环时 DeepLoop 与基线打成平手,一旦打开循环深度,验证损失和下游任务就持续拉开差距,而不是过去那种 train loss 涨、val loss 跑飞的尴尬。论文也强调一个关键区别——稳定循环深度需要按参数访问次数算缩放,不能只看名义层数。\n\n实际意义是,这规则几乎是免费工程改进:实现层只要把 DeepNorm 的 α、β 与 unrolled 深度 N 绑定,任何尝试把循环深度规模化进生产推理或 RL 后训练的团队都可以直接套用,不用重训基线对照。Looped Transformer 这条少参数多深度的路线,从论证理论可训变成工程可落地。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2607.13491","7437aeb9-930c-4866-a2e9-48003c1a792b",[10,14,17,20],{"id":11,"name":12,"slug":12,"description":13,"color":13},"5e628969-6d2a-437f-998a-104e4b16cfb1","ai-progress",null,{"id":15,"name":16,"slug":16,"description":13,"color":13},"40269b40-7942-4650-9672-ed2e6524d37a","ai-technology",{"id":18,"name":19,"slug":19,"description":13,"color":13},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":21,"name":22,"slug":22,"description":13,"color":13},"4f214978-cac1-4f39-aa4b-f92a0d0934b7","transformer","2026-07-17T22:13:48Z","2026-07-17T22:15:13.087060Z","2026-07-17T22:15:13.087072Z",true,"agent",2]