[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-reasoning-wait-marker-provenance-rl-training-data":3,"topics-all":38,"news-related-c780f08d-fa7e-4c4b-8cf5-ee367bde4c06":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"c780f08d-fa7e-4c4b-8cf5-ee367bde4c06","零 RL 涨 42.9 分几乎不说「等等」:它来自训练数据","港科大与华为诺亚实验室追踪 120 个开源模型训练端点后发现:零 RL 训练把 MATH-500 准确率中位数拉高 42.9 分,却几乎从不写出「等等」这个复核标记。这个词来自训练数据本身,RL 只是放大并磨损它;论文还发现两个 LLM 评判器中有一个把人工标注的趋势判反了。","在推理模型圈,\"等等\"(wait)早被当成 RL 教会模型的第一课:模型算到一半先冒出\"等等,这个对吗\",再回头检查一遍。港科大、华为诺亚实验室等团队把开源模型的 120 个训练端点翻了个遍,结论跟这个流行叙事不太一样——那个词几乎不是强化学习造出来的。[论文原文](https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.30882)\n\n## 涨了 42.9 分,却几乎不说\"等等\"\n\n最干净的对照组是 18 次零 RL 训练:底座本来不会说\"等等\",底座来自 9 个 Qwen2.5、Mistral、Qwen3 系列(0.5B 到 32B)。这些训练把 MATH-500 准确率中位数拉高 42.9 分,但\"等等\"的频率在 17\u002F18 次运行里始终低于每千 token 0.05 次,95% 置信上界都没越过 0.06。换成 AIME24、AIME25、AMC23、GSM8K、MMLU-Pro 也一样:从 Qwen2.5-7B 出发的零 RL,五个基准上没有一次说出\"等等\"超过 3 次。\n\n## 真正的出处是训练数据\n\n那么\"等等\"从哪来?数据。判定标准是每千 token 低于 0.05 次就算没有。OLMo-3 预训练阶段几乎为零(≤0.04 次\u002F千 token),到中期训练涨到 4.03 次,其中 88.6% 在句首——正好对应为 RL-Zero 引入思考轨迹那一步。14 个在推理轨迹上微调的模型里,13 个从不会说\"等等\"的底座学到了它。数据里本就带这个词的底座也一样:MiMo-7B-Base 每千 token 4.95 次,SmolLM3-3B-Base 4.18 次。\n\n## RL 做的是放大和磨损\n\n底座已经会说的话,两个公开的 RL-Zero 配方会把它放大:OLMo-3 RL-Zero 从 4.9 次\u002F千 token 涨到第 1900 步的峰值 17.0,最终检查点 12.4(6 个解码种子下 12.68 ± 1.55);MiMo-7B 从 4.95 涨到 6.62。同一底座换 KL-free GRPO 就不涨。形式还在磨损:规范写法\"Wait,\"占比从 88.9% 掉到 37.5%,叠写(\"Wait, wait\")从每千次 117.6 涨到 213.0,标记开头的句子从 11.8 个词缩到 6.1 个。论文对照语言学的\"语法化\":标记进入晚期阶段,形体被磨掉,功能被绑死在固定位置。\n\n## 它还是不是\"怀疑\"的信号\n\nRL 放大时,有没有顺手改变词的含义?论文用注入算术错误来测:接一段正确步骤,和一段中间被塞了错的步骤,看标记能否区分。RL-Zero 之后区分度 d′ 从 0.57 升到 1.09(95% CI 0.97–1.24),但判定标准从 0.11 掉到 −0.24——它更敏感,也更爱报警:\"等等\"单独看,d′ 从 0.77 升到 1.29,能跟住 73.7% 的出错步骤(底座 47.1%),也跟住 25.4% 没错的步骤(底座 19.9%)。而 SFT 之后再 RL,几乎改不动这个报警器。\n\n## 更该警惕的:评判器把趋势判反了\n\n论文顺手做了 60 句盲审人工标注审计,两个 LLM 评判器里有一个把人工共识的方向判反。主要评判器与共识一致性 κ=0.64,看着不低,但在同一批加权样本上,它给出的\"修订率\"从 26.0% 降到 12.9%,人工共识的修订率却从 1.3% 升到 15.9%——完全相反。另一个评判器 κ=0.71,方向跟得上人工(1.5% → 12.9%)。\n\n## 所以呢\n\n这是一把尺子被误用的故事。大家习惯在训练日志里数\"等等\"\"hmm\"的次数,当它反思能力变强的证据,再配一个 LLM 评判器打行为标签。但证据链是:这个标记得不得出现,取决于训练数据里本来有没有;RL 更多是在既有材料上放大和磨损,不是从零发明;它\"代表怀疑\"的含义,也随配方和检查点漂移。\n\n所以下次看到某个模型\"突然开始频繁反思\",第一反应或许不该是\"RL 教会了它\",而是\"它的数据里本来就这么写\"。至于这判断能撑多远,论文自己也提醒:证据是观察性的,18 次零 RL 只覆盖 0.5B–32B、英文数学与结果奖励,前沿规模 RL、过程奖励和其他语言都还没验证。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.30882","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"5e628969-6d2a-437f-998a-104e4b16cfb1","ai-progress",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"40269b40-7942-4650-9672-ed2e6524d37a","ai-technology",{"id":19,"name":20,"slug":20,"description":14,"color":14},"120fa59a-ff6f-4537-9bf5-f818df636a0e","benchmark",{"id":22,"name":23,"slug":23,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"af91d7e9-4c87-4631-a4ea-fefa8188c3a7","en","Reasoning Models' 'Wait' Came From Training Data, Not RL","Zero-RL lifted MATH-500 by a median 42.9 points yet rarely wrote 'wait'. Over 120 checkpoints the marker comes from training data, not RL.","In reasoning-model circles, \"wait\" has long been credited to reinforcement learning. A model writes \"Wait, is that correct?\" before re-checking a step, and the assumption was that RL taught it that habit — the visible sign of reasoning being acquired. Researchers from HKUST, Huawei Noah's Ark Lab and Peking University traced 120 training endpoints of open models and reached a different conclusion. [Paper](https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.30882)\n\n## 42.9 points of accuracy, and almost no \"wait\"\n\nThe cleanest control is zero-RL: the model trains only on its own sampled solutions, with verifiable rewards applied to the final answer. The team ran 18 such runs starting from 9 bases spanning Qwen2.5, Mistral and Qwen3 (0.5B to 32B) that had no marker to begin with. Accuracy on MATH-500 rose by a median of 42.9 points. And in 17 of those 18 runs the marker stayed below 0.05 occurrences per 1k tokens, with the exact 95% upper bound never crossing 0.06.\n\nThe pattern survives beyond a single benchmark. On AIME24, AIME25, AMC23, GSM8K and MMLU-Pro, zero-RL from Qwen2.5-7B never writes the word more than 3 times. By contrast, OLMo-3 RL-Zero raises it on every benchmark, from 4.62 to 17.10 per 1k on GSM8K, and only under MMLU-Pro's official few-shot prompt does it barely register at all (0.03 to 0.22 per 1k).\n\n## The marker arrives with the data\n\nSo where does it come from? Training data. The threshold for \"does not have it\" is 0.05 occurrences per 1k tokens. In OLMo-3 the word is essentially absent after pretraining (at most 0.04 per 1k), then rises to 4.03 after mid-training, with 88.6% of occurrences opening their sentence — right at the point where thinking traces were folded in for RL-Zero. Bases whose pretraining mixture already contains reasoning traces write it too: MiMo-7B-Base at 4.95 per 1k and SmolLM3-3B-Base at 4.18.\n\nFine-tuning reproduces the same thing. Of 14 models fine-tuned on reasoning traces from marker-free bases, 13 end up writing the marker at 0.5 per 1k or more. Marker forms track whatever the traces contained: OpenReasoning-Nemotron-7B opens sentences with \"but\" (98.3 per 1k) and \"perhaps\" (28.1 per 1k) instead of \"wait\" (0.2 per 1k).\n\n## What RL actually does: multiply, and wear down\n\nWhere the base already uses the marker, both released RL-Zero recipes multiply it. Along OLMo-3 RL-Zero the rate rises from 4.9 to a peak of 17.0 per 1k at step 1900, ending at 12.4 at the released checkpoint (12.68 ± 1.55 over 6 decoding seeds). MiMo-7B goes from 4.95 to 6.62. A KL-free GRPO run from the same base does not. RL after supervised fine-tuning also amplifies it in smaller steps: Phi-4 goes 1.56 to 2.58, DeepScaleR 3.65 to 4.07, Light-R1 2.79 to 3.21.\n\nThe form erodes at the same time. Position stays put — 89.6% of occurrences open their sentence at the base and 89.4% do at the end — but the canonical \"Wait,\" share falls from 88.9% to 37.5% as the comma drops. Reduplicated chains (\"Wait, wait\") rise from 117.6 to 213.0 per 1k occurrences, and marker-initial sentences shrink from 11.8 to 6.1 words. The paper reads this through grammaticalization: a late-stage marker loses substance and binds to fixed position. Usage drifts too, with a Jensen-Shannon divergence of 0.05 between the last checkpoint and the base, against a noise floor of 0.002.\n\n## Does the word still mean doubt?\n\nMultiplying the word did not automatically change what it means. The team built a test that needs no labels: take a prefix of correct steps, and a parallel copy with one injected arithmetic error, then check whether a revision marker appears within two sentences. The measure is d′, how well the marker separates errors from correct steps.\n\nAlong RL-Zero the marker becomes both more sensitive and more liberal: d′ rises from 0.57 to 1.09 (95% CI [0.97, 1.24]), but the criterion falls from 0.11 to −0.24. Taken alone, \"wait\" sees d′ rise from 0.77 to 1.29 (CI [1.16, 1.44]) while c′ falls from 0.46 to 0.01. It follows 73.7% of error prefixes, up from 47.1% at the base — but also 25.4% of error-free ones, up from 19.9%. After SFT, RL leaves the alarm almost unchanged: the criterion moves from −0.14 to −0.19 at 7B, and d′ from 0.82 to 1.02.\n\n## The sharper warning: a judge reversed the trend\n\nThe paper also ran a 60-sentence blind audit with expert consensus labels on sentences drawn from the base and the final RL-Zero checkpoint. The primary LLM judge agrees with consensus at κ = 0.64 — a respectable number. Yet on the same weighted items its revision rate falls from 26.0% to 12.9%, while the human consensus rate rises from 1.3% to 15.9%. It gets the direction backwards. A second judge tracks consensus correctly (1.5% to 12.9%, κ = 0.71).\n\n## So what\n\nThis reads as a story about a measuring instrument used carelessly. Counting \"wait\" and \"hmm\" in training logs has become standard evidence that reflection is increasing, and an LLM judge is often bolted on to label the behaviour. The evidence chain here says otherwise: whether the marker appears at all depends on whether the training data already contained it; RL mostly multiplies and wears down inherited material rather than inventing it; and the marker's evidential weight shifts with the recipe and the checkpoint.\n\nSo the next time a model starts \"reflecting\" a lot, the first question should perhaps not be \"what did RL teach it\" but \"was this already in the data\". The authors are careful about how far the finding travels: the evidence is observational, the zero-RL result rests on 18 runs over 0.5B–32B models, English mathematics and outcome rewards, and frontier-scale RL, process rewards and other languages all remain untested. Still, for anyone building reasoning models, the actionable part is clear — measure the marker directly on the checkpoint you care about, and never let an automated judge stand in for a human consensus without validating it.","reasoning-wait-marker-provenance-rl-training-data","2026-10-25T05:15:00Z","2026-10-25T05:16:55.913177Z","2026-10-25T05:16:55.913184Z",true,"agent",17,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"c368ad9f-9308-4a0b-8f5c-3ae4601b48b9","D2K-Bench: 专家设计把 LLM 写 GPU 核提速 33.9%","d2k-bench-llm-gpu-kernel-design-guidance","2026-10-07T03:00:00+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"43eda321-b0b7-4df7-b20e-9758cbab42c9","记忆越完整,眼前题越做不对:MemTrapBench 把 LLM 长期记忆框架打回原形","memtrapbench-llm-memory-cognitive-traps","2026-08-22T04:00:00+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"88105269-9641-44c3-a705-1cf07314614f","LLM 思维链能看出\"用了几分力\":SARE 给每一步推理做 CT 扫描","step-aware-reasoning-energy-llm-cot","2026-08-04T04:00:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"95b04c15-d5e5-4dca-ab2b-14e343bdd4e6","UC Berkeley 曝光 AI 基准测试系统性漏洞：45 种方法可在 13 个主流榜单上「不解决任何问题拿满分」","uc-berkeley-benchmark-45-cheats-13-leaderboards","2026-05-15T01:00:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"aa294551-fbef-4cf4-b466-1928ab1925bb","朴素 RAG 拿下 6\u002F7 组评测:复杂记忆系统被比下去了","naive-rag-beats-complex-agentic-memory","2026-10-25T07:17:45+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"b809f974-e141-4509-96c9-f71e8570a4a0","System1-Arena:30 模型同场比校准","system1-arena-decision-model-benchmark","2026-10-16T03:00:00+00:00"]