[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-princeton-ucla-deeploop":3,"news-related-f8ea285b-f717-4f82-aafd-6a096ca6cf46":36},{"id":4,"title":5,"summary":6,"content":6,"original_url":7,"source_id":8,"tags":9,"translations":23,"news_slug":29,"published_at":30,"created_at":31,"modified_at":32,"is_published":33,"publish_type":34,"image_url":13,"view_count":35},"f8ea285b-f717-4f82-aafd-6a096ca6cf46","DeepLoop：Princeton\u002FUCLA 修对 Looped Transformer 残差缩放","Looped Transformer 想用一份共享参数、多轮迭代去放大模型深度,一直被一个隐形问题卡住:每次回访都让这条残差分支吃到相同更新,前向-反向耦合关系和 untied Transformer 不一样,DeepNorm 那套按层数取 α=β 的经验法则直接失灵,损失曲线在循环层数一上去就抖动甚至发散。\n\nPrinceton 王梦迪和 UCLA 顾全泉团队(arXiv 2607.13491,2026-07-15)给出第一性原理答案。论文把参数被访问 k 次翻译成一个一阶扰动界,引入 visit-alignment coefficient κ_R:在解相关区域 κ_R 退化,界恢复成 DeepNorm 的 1\u002F4 指数;但在保守对齐区域——也就是工程里更常见的状态——指数必须从 1\u002F4 抬到 1\u002F2,随循环数 N 增长。DeepLoop 的核心:沿用 Post-LN DeepNorm 骨架,只把残差缩放系数改成 α=(2N)^{1\u002F2}、β=(8N)^{-1\u002F2},计算量几乎没变,循环深度第一次能稳住。\n\nGPT-2 small\u002Fmedium 上的实验很直接:不循环时 DeepLoop 与基线打成平手,一旦打开循环深度,验证损失和下游任务就持续拉开差距,而不是过去那种 train loss 涨、val loss 跑飞的尴尬。论文也强调一个关键区别——稳定循环深度需要按参数访问次数算缩放,不能只看名义层数。\n\n实际意义是,这规则几乎是免费工程改进:实现层只要把 DeepNorm 的 α、β 与 unrolled 深度 N 绑定,任何尝试把循环深度规模化进生产推理或 RL 后训练的团队都可以直接套用,不用重训基线对照。Looped Transformer 这条少参数多深度的路线,从论证理论可训变成工程可落地。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2607.13491","7437aeb9-930c-4866-a2e9-48003c1a792b",[10,14,17,20],{"id":11,"name":12,"slug":12,"description":13,"color":13},"5e628969-6d2a-437f-998a-104e4b16cfb1","ai-progress",null,{"id":15,"name":16,"slug":16,"description":13,"color":13},"40269b40-7942-4650-9672-ed2e6524d37a","ai-technology",{"id":18,"name":19,"slug":19,"description":13,"color":13},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":21,"name":22,"slug":22,"description":13,"color":13},"4f214978-cac1-4f39-aa4b-f92a0d0934b7","transformer",[24],{"id":25,"lang":26,"title":27,"summary":28,"content":28},"71796e32-6e7e-4df4-ae76-dd13edc3a6b5","en","DeepLoop: Princeton\u002FUCLA fix Looped Transformer residual scaling","Looped Transformer tries to use one set of shared parameters, multiple rounds of iteration, to amplify model depth — but has always been blocked by an invisible problem: every revisit has this residual branch eating the same update, the forward-backward coupling differs from untied Transformer, and the empirical rule of DeepNorm taking α=β by layer count fails directly, with the loss curve jittering or even diverging when loop depth goes up. Princeton's Mengdi Wang and UCLA's Guanquan Cao team (arXiv 2607.13491, 2026-07-15) gives a first-principles answer. The paper translates the parameter being visited k times into a first-order perturbation bound, introducing the visit-alignment coefficient κ_R: in the decorrelated region κ_R degenerates, the bound recovers DeepNorm's 1\u002F4 exponent; but in the conservative-alignment region — the more common state in engineering — the exponent must be lifted from 1\u002F4 to 1\u002F2, growing with loop count N. The core of DeepLoop: keep the Post-LN DeepNorm skeleton, just change the residual scaling coefficients to α=(2N)^{1\u002F2} and β=(8N)^{-1\u002F2}, with almost no extra compute, and loop depth can finally be stable. The experiments on GPT-2 small\u002Fmedium are direct: when not looping, DeepLoop ties the baseline; once loop depth is turned on, validation loss and downstream tasks keep pulling apart, rather than the previous embarrassment of training loss going up while val loss ran away. The paper also emphasizes a key distinction — stable loop depth requires scaling by the number of parameter visits, not just the nominal layer count. The practical significance is that this rule is almost free engineering improvement: at the implementation layer, just bind DeepNorm's α, β to the unrolled depth N, and any team trying to scale loop depth into production inference or RL post-training can apply it directly without retraining a baseline. The Looped Transformer line, which uses few parameters for more depth, moves from \"demonstrably trainable in theory\" to \"engineering-deployable\".","princeton-ucla-deeploop","2026-07-17T22:13:48Z","2026-07-17T22:15:13.087060Z","2026-08-19T02:08:40.142862Z",true,"agent",141,{"items":37},[38,43,48,53,58,63],{"id":39,"title":40,"news_slug":41,"published_at":42},"cb7fb8b3-5862-4cba-adab-c4794e989966","图灵奖得主 Pearl 长访谈：LLM 能讲因果只是因为人类替它爬过了因果阶梯","judah-pearl-llm-causal-ladder-agi","2026-07-31T07:00:00+00:00",{"id":44,"title":45,"news_slug":46,"published_at":47},"f8436dd3-d6fc-4ea7-9f2e-1086026c11d0","Transformer 的几何之眼：arXiv 2607.17146 把注意力炼成薛定谔桥，把 SGD 写成伊藤扩散","transformer-geometry-schrodinger-bridge","2026-07-23T12:10:00+00:00",{"id":49,"title":50,"news_slug":51,"published_at":52},"e54f030e-14ed-4262-9dd9-8685fdbb03ab","DiscoLoop 把循环 Transformer 的「表征瓶颈」焊死:双通道架构让多跳推理一步到位","discoloop-dual-channel-recurrent","2026-07-20T08:00:00+00:00",{"id":54,"title":55,"news_slug":56,"published_at":57},"43eda321-b0b7-4df7-b20e-9758cbab42c9","记忆越完整,眼前题越做不对:MemTrapBench 把 LLM 长期记忆框架打回原形","memtrapbench-llm-memory-cognitive-traps","2026-08-22T04:00:00+00:00",{"id":59,"title":60,"news_slug":61,"published_at":62},"22a1a718-0eb6-46e5-8ee8-825400de11d1","DeepMind WeatherNext 在 Nature 发论文：用 28 km 粗分辨率做出多一天的飓风预警,代码权重全部开源","deepmind-weathernext-cyclones-nature-open-source","2026-08-10T02:00:00+00:00",{"id":64,"title":65,"news_slug":66,"published_at":67},"49d19ba1-8f45-475c-bed1-a69dc353523e","字节跳动用 10 万亿参数下注：规模赛跑与张一鸣的「不蒸馏」表态","bytedance-10t-mythos-zhangyiming-no-distill-2026-08","2026-08-08T00:00:00+00:00"]