[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-jpmorgan-factorial-retest-llm-ingredient-gains":3,"topics-all":38,"news-related-f02185e7-5f4c-46c6-bb18-575fbf3b1c0a":67},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"f02185e7-5f4c-46c6-bb18-575fbf3b1c0a","摩根大通实测:五项前沿配方,四项真有用","摩根大通 LLM Suite 团队用 16 组组合、两个随机种子的因子实验复测五项流行改进: 专家路由 -0.0983、Muon -0.0506、Gated DeltaNet 混合 -0.0365、额外归一化 -0.0194 nats, 注意力门控平均约零; 增益大体可叠加, 但统计口径与混叠会反转单项结论.","每次新模型发布, 配方单上总列着几项「带来的改进」: 更好的注意力、更省的优化器、更聪明的路由. 但每项改进都是对着自家 baseline 单独测的, 换个环境叠在一起, 还剩多少? 摩根大通 LLM Suite 团队的 Biswa Sengupta 把这个问题做成了严格实验, 论文 11 月 18 日挂上 arXiv (https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.24540), 页首声明这是信息性文档、非投研产品.\n\n## 同一个模型, 五个开关\n\nbaseline 是个 8 层、51.5M 参数的标准 decoder, 在 FineWeb 上按 3 亿 token 预算训练. 上面挂五个可开关的改动: E 把第 2-8 层换成 32 专家 top-3 路由, 总参数 161.3M、激活 51.6M; M 把隐层权重矩阵交给 Muon 更新, 其余参数仍用 AdamW; H 把第 1-3、5-7 层换成 Gated DeltaNet 线性记忆, 只留第 4、8 两层全局注意力 (这两层连位置编码都不用); S 给每个子层的输入和输出都加归一化; G 给注意力输出加 sigmoid 门控, 同时收窄前馈层来对齐激活参数量.\n\n实验设计本身是亮点: 经典分数因子设计, 16 个组合、每个组合换随机种子重复一次, 共 32 次核心训练、约 96 亿 token. 学习率按优化器家族各自扫过: AdamW 峰值 0.002, Muon 峰值 0.016, 后者每次更新跑 5 次 Newton-Schulz 迭代. 同种子的组合喂完全相同的批次 (用首批哈希校验), 最终损失在 1049 万个 held-out token 上计算.\n\n## 结果: 四项真有用, 一项约等于零\n\n按两种子平均的损失对比 (nats, 负值代表有利): 专家路由 E -0.0983, Muon -0.0506, Gated DeltaNet 混合 -0.0365, 额外归一化 -0.0194, 都越过 ±0.0043 的参考带; 注意力门控 G 是 +0.0018, 落在带内, 平均意义上没有收益. 单开关对比在两个种子之间的差异不超过 0.004 nats. 五项全开的组合在两个种子下都是选定集合里损失最低 (3.6792 与 3.6750). 两个种子间 15 组对比的相关系数 0.996, 复测相当稳.\n\n## 两个反转点\n\n第一个是 G 的故事. 单独把 G 加到全关 baseline 上, seed-0 损失降了 0.019 nats, 看着像有效; 放进组合里复测, 平均值变成 +0.0018. 论文也老实交代: 这不能证明门控本身没用, 因为 G 为对齐参数量收窄了前馈层, 也可能是别的改动削弱了它的收益. 而提出这个门控的原始论文在 15B MoE 和 1.7B 稠密模型、3.5 万亿 token 的设置里测得它稳定带来增益 (https:\u002F\u002Farxiv.org\u002Fabs\u002F2505.06708) — 同一个部件, 量级和预算不同, 结论可以完全反过来.\n\n第二个是统计口径. 初始用 Lenth 伪标准误估计噪声约 0.020 nats, 只有 E、M、H 显著; 换成重复实验估出的 0.0036 (小了约 6 倍) 后, S 和若干交互项也浮出水面. 一开始被当成噪声的信号, 是真实存在的. 交互项也不可忽略: 最大的成对交互 MH\u002FGSE 达 +0.0122 nats, 加上参考带约 0.017, 贴着论文自设的 0.02 nats 实用界.\n\n## 该泼的冷水\n\n- 量级: 51.5M 激活参数、单一语料、1024 token 上下文、3 亿 token 预算, 训练时长、能耗与推理速度也没有对齐, 结论不能外推到前沿大模型.\n- E 的收益混着约 3 倍的总容量 (161.3M 对 51.5M), 不是纯路由技巧的贡献.\n- 分辨率 V 设计有混叠: G 的对比里混着四开关联合效应, 要彻底拆开需要全部 32 个组合, 这里只跑了 16 个.\n- 验证集同时用于调参和最终比较, 属于开发集结果, 不是新测试集.\n- 开发中还抓到两个数值 bug: 一个 RMSNorm GPU kernel 会泄漏未来 token (更小宽度的测试漏掉了它), 一个 sigmoid 路由在极端 logits 下梯度下溢. 修复前的低损失是假的.\n\n## 所以呢\n\n优化器这条线上, 独立证据方向一致: Moonshot 的 Moonlight 把 Muon 扩到 16B 规模, 缩放实验给出约 2 倍计算效率 (https:\u002F\u002Farxiv.org\u002Fabs\u002F2502.16982); 另一支团队系统评测十种优化器后也警告, 排名严重依赖调参和评测设置 (https:\u002F\u002Farxiv.org\u002Fabs\u002F2509.02046). 作者的落点也写在摘要里: 先把有希望的组件放在一起测, 再重复比较, 然后才决定留下哪些. 这篇论文的方法论价值大于任何单个数字: 别按论文清单逐项打勾, 把候选组件放进同一个实现、同一份预算里对比, 至少跑两个种子 — 单开关、单种子的「有效」, 可能只是噪声、混叠, 或者一个还没被发现的 bug.","https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.24540","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"120fa59a-ff6f-4537-9bf5-f818df636a0e","benchmark",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",{"id":19,"name":20,"slug":20,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":22,"name":23,"slug":23,"description":14,"color":14},"d11f0044-8aef-487c-bebe-89ce4683a4a3","moe",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"788e21ab-0665-4354-893d-5ffbd322ca5a","en","Five Frontier Recipes, One 51M Model: Four Actually Help","A JPMorgan factorial retest of five LLM upgrades in one 51M model: four cut loss, an attention gate adds about zero, and gains mostly stack.","Every model launch ships with a list of claimed improvements: better attention, a leaner optimizer, smarter routing. Each item was measured against its own baseline, though. Stack them in a different setting and how much survives? Biswa Sengupta of the LLM Suite team at JPMorgan Chase turned that question into a disciplined experiment; the paper hit arXiv on November 18 (https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.24540), with a front-page disclaimer that it is informational and not a research-department product.\n\n## One model, five switches\n\nThe baseline is an eight-layer, 51.5M-parameter decoder trained on FineWeb under a nominal 300M-token budget. Five switches sit on top: E swaps layers 2-8 for 32-expert top-3 routing, with 161.3M total and 51.6M active parameters; M hands the hidden weight matrices to Muon while the rest stay on AdamW; H turns layers 1-3 and 5-7 into Gated DeltaNet linear memory, keeping only layers 4 and 8 on global attention (with no positional encoding at all); S adds normalisation to both the input and the output of each sub-layer; G inserts a sigmoid gate on attention outputs while narrowing the feed-forward layers to keep active parameters matched.\n\nThe design is the highlight: a classical fractional factorial layout, 16 combinations, each repeated with a second random seed - 32 core runs, about 9.6B tokens. Learning rates were swept per optimizer family: peak 0.002 for AdamW and 0.016 for Muon, with five Newton-Schulz iterations per update. Configurations sharing a seed receive identical batches, checked by first-batch hashes, and the final loss is measured on 10,485,760 held-out tokens.\n\n## Four help, one does nothing on average\n\nBy the two-seed mean loss contrast (nats, negative favors the switch): routing E -0.0983, Muon -0.0506, Gated DeltaNet hybrid -0.0365, extra normalisation -0.0194, all beyond the ±0.0043 reference margin. The attention gate G lands at +0.0018, inside the band - no average benefit. Single-switch contrasts differ by at most 0.004 nats between seeds. The all-five configuration posts the lowest loss in the selected set at both seeds (3.6792 and 3.6750). Across the 15 contrasts, the two seeds correlate at 0.996, so the replication is stable.\n\n## Two plot twists\n\nFirst, G. Added alone to the all-off baseline, it cuts seed-0 loss by 0.019 nats and looks effective; inside the selected combinations its repeated mean is +0.0018. The paper is candid: this does not prove the gate useless, because G narrowed the feed-forward layers to match parameters, and some other change may have eaten its benefit. The original gated-attention paper measured steady gains at 15B MoE and 1.7B dense scale over 3.5 trillion tokens (https:\u002F\u002Farxiv.org\u002Fabs\u002F2505.06708) - the same component flips conclusion when the scale, budget and parameter bookkeeping change.\n\nSecond, the noise estimate. Lenth's pseudo-standard error implies a run standard deviation near 0.020 nats, under which only E, M and H separate from zero. The repeated-run estimate is 0.0036, about six times smaller, and under it S plus several interactions emerge. Signals initially filed as noise turn out to be real. Interactions matter too: the largest pairwise contrast MH\u002FGSE reaches +0.0122 nats, about 0.017 with its margin, brushing the paper's own 0.02-nat practical bound.\n\n## The cold water\n\n- Scale: 51.5M active parameters, one corpus, a 1,024-token context, a 300M-token budget; training time, energy and inference speed are not matched - no extrapolation to frontier models.\n- E's gain bundles roughly three times the total capacity (161.3M vs 51.5M); it is not pure routing skill.\n- The resolution-V design aliases: the G contrast mixes in a four-switch effect; unaliasing everything would need all 32 combinations, and only 16 were run.\n- The validation set served both tuning and final comparisons, so these are development results, not a fresh test set.\n- Two numerical bugs were caught during development: an RMSNorm GPU kernel leaked future-token information that smaller-width tests missed, and a sigmoid router underflowed gradients on extreme logits. Pre-fix losses were invalid.\n\n## So what\n\nOn the optimizer line the independent evidence points the same way: Moonshot's Moonlight scaled Muon to 16B and reports roughly 2x computational efficiency in scaling-law experiments (https:\u002F\u002Farxiv.org\u002Fabs\u002F2502.16982), and a separate team that benchmarked ten optimizers warns that rankings hinge on tuning and evaluation setup (https:\u002F\u002Farxiv.org\u002Fabs\u002F2509.02046). The paper's own abstract lands on the process lesson: test promising components together, repeat the comparisons, then decide what to keep. The methodology matters more than any single number here: stop ticking items off paper checklists. Put candidate components into one implementation under one budget and repeat the comparison across seeds - a single switch on a single seed can be noise, aliasing, or a bug nobody has found yet.","jpmorgan-factorial-retest-llm-ingredient-gains","2026-11-19T17:25:00Z","2026-11-19T17:09:34.729289Z","2026-11-19T17:09:34.729299Z",true,"agent",94,[39,49,58],{"slug":40,"tag_slug":41,"title_zh":42,"title_en":43,"intro_zh":44,"intro_en":45,"id":46,"is_active":35,"created_at":47,"modified_at":48},"image-generation","text-to-image","图像生成：从文生图到逐框编辑的模型竞赛","Image Generation: from text-to-image to frame-level editing","文生图已是红海，战场正转向编辑与可控性：FLUX.2\u002F3 用开源和量化把生图搬进消费级显卡，Qwen-Image、Seedream 等国产模型紧咬不放，GPT Image 2 则把推理能力带进像素。本专题追踪图像生成与编辑模型的发布、架构演进、加速量化与商业化数据，看「好图」如何变成「可用的图」。","Text-to-image is a crowded race, and the frontier has moved to editing and control: FLUX.2\u002F3 push open weights and quantization onto consumer GPUs, Qwen-Image and Seedream close in from China, and GPT Image 2 brings reasoning into pixels. This topic tracks model releases, architecture shifts, acceleration work, and the business numbers behind \"pretty\" becoming \"usable\".","52de7c59-7864-4569-8ae7-3d656d26a1a1","2026-11-20T07:20:23.534506Z","2026-11-20T07:20:23.534527Z",{"slug":50,"tag_slug":50,"title_zh":51,"title_en":52,"intro_zh":53,"intro_en":54,"id":55,"is_active":35,"created_at":56,"modified_at":57},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":59,"tag_slug":59,"title_zh":60,"title_en":61,"intro_zh":62,"intro_en":63,"id":64,"is_active":35,"created_at":65,"modified_at":66},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":68},[69,74,79,84,89,94],{"id":70,"title":71,"news_slug":72,"published_at":73},"c4623b85-37f5-435e-ae98-e67eb72eb1dc","别再逐篇问大模型:AI写程序当评委,每主题0.17美元起","execjudges-llm-written-program-judges-retrieval-eval","2026-11-20T01:40:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"8e741f04-82a2-46da-9d30-3c31117ca70c","Ambient SFT:让预训练的自己当高温老师","ambient-sft-temperature-matching","2026-11-18T17:30:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"d1d54032-073b-433e-9430-547c6ffb4a49","贵的模型，没写出更快的代码：9455 份实测","perfarena-llm-code-performance-ranking","2026-11-16T19:13:41+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"d9243227-680a-4fde-908d-8990969a54e0","LuthorSQL：7B 模型替 Agent 省三成账单","luthorsql-text-to-sql-agent-cascade","2026-11-16T05:00:00+00:00",{"id":90,"title":91,"news_slug":92,"published_at":93},"08c85d0c-1ad8-43d6-bd32-0ec8b21bf2aa","给教师开小抄,学生涨分却不是抄来的","opsd-privileged-solution-teacher-audit","2026-11-15T01:30:00+00:00",{"id":95,"title":96,"news_slug":97,"published_at":98},"1ec34a26-f012-4bc4-8193-14e651c07516","程序进化搜出记忆设计:涨12个百分点,成本只占35%","mape-agent-memory-program-evolution","2026-11-14T05:05:00+00:00"]