[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-nopo-negative-only-policy-optimization":3,"topics-all":38,"news-related-03cf5d4f-a220-499e-a100-4a64509d112d":67},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"03cf5d4f-a220-499e-a100-4a64509d112d","64.5%的「通过」其实是错的：RL该只信负样本","南洋理工大学与Skywork的NeurIPS 2026 Spotlight论文把RLVR的现实困境形式化为单边可验证性：验证器能可靠挑错，却无法证明对——TACO上通过公开测试的解答64.5%其实是错的。提出的NOPO只用可信负样本更新策略，平均分、大k推理与跨基准迁移全面超过GRPO。","RLVR（可验证奖励强化学习）这套训练配方里，藏着一个很少被点破的假设：验证器说得清对错。数学、代码任务好训，是因为答案可以机器判分；可一旦离开这些舒适区，完整的验证器往往造不出来，手头只剩公开测试、格式约束，或者让另一个模型来挑错。一篇被 NeurIPS 2026 接收为 Spotlight 的论文（[arXiv:2611.17287](https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.17287)，南洋理工大学与 Skywork AI 团队）把这种处境形式化为「单边可验证性」（OSV），解法也反直觉：只用负样本训练，不奖励任何「通过」。\n\n## 拒绝可信，通过存疑\n\n论文先量化了这种不对称。Qwen3-4B 在 TACO 上采样，通过公开测试的解答拿完整测试套件一对答案，64.5% 其实是错的（Qwen3-8B 为 66.8%）。原因很朴素：训练时平均每题只有 1.7 个公开测试，评测用的完整套件平均 204 个。CodeContests 上过检解答里的假阳性占 20.7%（1.9 个测试对 12 个），ZebraLogic 的约束检查是 73.0%，LLM 自查也有 46.3%。反过来，验证器说「错」几乎总是真错：公开测试和约束检查的负例精确率按构造是 100%，LLM 自查达 95.8%。\n\n论文的原话很扎心：假阳性一旦被当作奖励，就会变成策略优化动力学的吸引子——钻空子的解法只要混过部分检查就拿到正梯度，模型于是学会提高过检率，而不是提高正确率。\n\n## NOPO：把「通过」当未标注\n\n方法因此叫 Negative-Only Policy Optimization。验证器只出它可信的那部分证据：高置信负例。样本级用 NPO 式抑制损失，相对参考策略压低负例概率，压够了梯度自动衰减；组级加一道支持度门控：同一 prompt 的 16 条采样里若一条通过的都没有，这个组直接跳过，不做漫无方向的「远离」更新。通过检查的样本不充当正确示范，只贡献计数。整套训练不重：verl 加 vLLM，8 张 A100 跑约 12 小时，150 步。\n\n## 平均第一，大 k 才见真章\n\n论文实测，Qwen3-4B 五个基准平均 pass@16 从基座 46.85 涨到 52.86，GRPO 是 48.93；Qwen3-8B 上是 52.13 对 48.27。十个「基准×基座」格子里 NOPO 拿下九个。更硬的证据在大 k 与迁移：CodeContests 上 k=512 时 NOPO 66.95、基座 51.05，GRPO 只有 48.54——论文特意点名 GRPO 的 pass@1 全场最高（37.23），但 k=512 时在两个基准上都跌回基座之下，属于「锐化」而非扩边界。迁移上，CodeContests 训出的模型在未见基准平均 29.49，基座 23.66，GRPO 只有 23.30，连基座都不如。按 512 采样口径，NOPO 保住基座会做的 122 题中的 120 题，另新解出 40 题。\n\n还有一条 reward hacking 的直接证据：GRPO 训完后，过检解答的假阳性率从 20.7% 升到 25.2%，负类召回从 91.3% 掉到 79.8%；NOPO 把假阳性率压到 16.6%。\n\n## 机制层不是孤证\n\n「负样本才是主要有效信号」并非这一篇的独家发现。论文中的对比方法 w-Reinforce（[arXiv:2506.01347](https:\u002F\u002Farxiv.org\u002Fabs\u002F2506.01347v2)，Zhu 等人，NeurIPS 2025）独立报告：只用负样本训练，在 Qwen2.5-Math-7B、Qwen3-4B、Llama-3.1-8B-Instruct 上，k 到 256 常常追平或超过 PPO 与 GRPO；只强化正样本则提升 pass@1、拖累大 k，因为多样性被压掉。两个团队的分歧在处理方式：w-Reinforce 仍给通过样本 0.1 的小权重，NOPO 干脆全部当未标注。\n\n论文自述的边界也要交代：负样本只说「别去哪」，不说「该去哪」；OlympiadBench 上 Qwen3-8B 这一个格子，NOPO 输给 GRPO 0.45 点，作者归因于该场景负类召回只有 8.8%，可信负例太少。\n\n## 所以呢\n\n在自己攒验证器做 RL 的人，这篇的直接启示有两条：一，别把「没挑出错」写成「做对了」，那等于给钻空子的解法发奖金；二，评估别只盯 pass@1，大 k 和跨基准迁移才分得清模型是真变强，还是仅仅变得会过检。便宜的检查做可靠的拒绝，不确定的接受晾在一边——单边验证器不是残缺的验证器，而是一类还没被用满的信号源。\n","https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.17287","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"5e628969-6d2a-437f-998a-104e4b16cfb1","ai-progress",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"120fa59a-ff6f-4537-9bf5-f818df636a0e","benchmark",{"id":19,"name":20,"slug":20,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":22,"name":23,"slug":23,"description":14,"color":14},"c187600e-804c-4697-b828-1e4330e0eb10","qwen",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"270d7ea0-9263-46a6-860f-3da3e3120c4a","en","When checks can only reject: negative-only RL beats GRPO","NeurIPS 2026 Spotlight: on TACO, 64.5% of public-test passes are wrong. NOPO trains RL only on trusted verifier negatives, beating GRPO at k=512.","RLVR - reinforcement learning with verifiable rewards - quietly assumes the verifier can tell right from wrong. Math and code are easy precisely because answers can be machine-checked. Outside those comfort zones, a complete verifier is often impossible to build: all you have is public tests, format constraints, or another model looking for flaws. A NeurIPS 2026 Spotlight paper from Nanyang Technological University and Skywork AI ([arXiv:2611.17287](https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.17287)) formalizes this predicament as one-sided verifiability (OSV), and its fix is counterintuitive: train only on negatives, reward no 'pass' at all.\n\n## Rejections are trustworthy, acceptances are not\n\nThe paper first measures the asymmetry. Sampling with Qwen3-4B on TACO, 64.5% of solutions that pass the public tests turn out wrong against the full test suite (66.8% for Qwen3-8B). The reason is plain: training sees an average of 1.7 public tests per problem, while evaluation uses a full suite averaging 204. On CodeContests the false-positive share among accepted solutions is 20.7% (1.9 tests vs 12); ZebraLogic constraint checks sit at 73.0%, and an LLM self-verifier at 46.3%. In the other direction, a 'rejected' verdict is almost always genuinely wrong: negative precision is 100% by construction for tests and constraints, and 95.8% for the LLM self-verifier.\n\nThe paper's phrasing is blunt: once false positives are used as rewards, they become attractors of the policy optimization dynamics - shortcut solutions that slip past partial checks collect positive gradients, so the model learns to raise its pass rate instead of its correctness.\n\n## NOPO: treat 'passed' as unlabeled\n\nThe method, Negative-Only Policy Optimization, uses the verifier only for the evidence it can safely give: high-confidence negatives. Per sample, an NPO-style suppression loss pushes down negative responses relative to a reference policy, with a gradient that self-attenuates once suppression is done. Per group, a support gate skips any prompt whose 16 rollouts contain zero non-falsified responses, avoiding directionless 'move-away' updates. Passed samples are never treated as correct demonstrations; they only count. Training is light: verl plus vLLM, eight A100 GPUs, about 12 hours, 150 steps.\n\n## First on average, and the truth shows at large k\n\nPer the paper's own measurements, average pass@16 across five benchmarks rises from 46.85 (base) to 52.86 on Qwen3-4B, with GRPO at 48.93; on Qwen3-8B it is 52.13 versus 48.27. NOPO takes nine of ten benchmark-backbone cells. The harder evidence is at large k and in transfer: on CodeContests at k=512, NOPO reaches 66.95 and the base 51.05, while GRPO manages only 48.54. The paper notes GRPO has the best pass@1 of all methods (37.23) yet falls below the base model at k=512 on both benchmarks - sharpening, not frontier expansion. In transfer, a model trained on CodeContests averages 29.49 on unseen benchmarks (base 23.66, GRPO 23.30 - below base). At the 512-sample budget NOPO keeps 120 of the 122 problems the base solves and adds 40 new ones.\n\nThere is also direct reward-hacking evidence: after GRPO training, the false-positive share among accepted solutions rises from 20.7% to 25.2% while negative recall drops from 91.3% to 79.8%; NOPO pushes the false-positive share down to 16.6%.\n\n## The mechanism is not a one-off\n\n'Negatives carry the useful signal' is not this paper's private discovery. The baseline it compares against, w-Reinforce ([arXiv:2506.01347](https:\u002F\u002Farxiv.org\u002Fabs\u002F2506.01347v2), Zhu et al., NeurIPS 2025), independently reports that training with only negative samples - on Qwen2.5-Math-7B, Qwen3-4B and Llama-3.1-8B-Instruct, with k up to 256 - often matches or surpasses PPO and GRPO, while reinforcing only correct responses lifts pass@1 but hurts large k as diversity collapses. The two teams differ in treatment: w-Reinforce still gives passed samples a small weight of 0.1; NOPO treats them all as unlabeled.\n\nThe paper's own limits deserve mention: negatives say where not to go, not where to go. On OlympiadBench with Qwen3-8B - the one cell NOPO loses - it trails GRPO by 0.45 points, which the authors attribute to negative recall of only 8.8% there: too few trusted negatives.\n\n## So what\n\nIf you are building your own verifier for RL, two takeaways. First, do not equate 'no error found' with 'correct' - that is a bonus program for shortcut solutions. Second, do not judge by pass@1 alone; large-k and cross-benchmark transfer are what separate genuinely stronger models from merely better test-passers. Cheap checks make reliable rejections; uncertain acceptances can stay on the shelf. A one-sided verifier is not a broken verifier - it is an underused signal source.\n","nopo-negative-only-policy-optimization","2026-11-14T21:30:00Z","2026-11-14T21:07:32.547304Z","2026-11-14T21:07:32.547314Z",true,"agent",186,[39,49,58],{"slug":40,"tag_slug":41,"title_zh":42,"title_en":43,"intro_zh":44,"intro_en":45,"id":46,"is_active":35,"created_at":47,"modified_at":48},"image-generation","text-to-image","图像生成：从文生图到逐框编辑的模型竞赛","Image Generation: from text-to-image to frame-level editing","文生图已是红海，战场正转向编辑与可控性：FLUX.2\u002F3 用开源和量化把生图搬进消费级显卡，Qwen-Image、Seedream 等国产模型紧咬不放，GPT Image 2 则把推理能力带进像素。本专题追踪图像生成与编辑模型的发布、架构演进、加速量化与商业化数据，看「好图」如何变成「可用的图」。","Text-to-image is a crowded race, and the frontier has moved to editing and control: FLUX.2\u002F3 push open weights and quantization onto consumer GPUs, Qwen-Image and Seedream close in from China, and GPT Image 2 brings reasoning into pixels. This topic tracks model releases, architecture shifts, acceleration work, and the business numbers behind \"pretty\" becoming \"usable\".","52de7c59-7864-4569-8ae7-3d656d26a1a1","2026-11-20T07:20:23.534506Z","2026-11-20T07:20:23.534527Z",{"slug":50,"tag_slug":50,"title_zh":51,"title_en":52,"intro_zh":53,"intro_en":54,"id":55,"is_active":35,"created_at":56,"modified_at":57},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":59,"tag_slug":59,"title_zh":60,"title_en":61,"intro_zh":62,"intro_en":63,"id":64,"is_active":35,"created_at":65,"modified_at":66},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":68},[69,74,79,84,89,94],{"id":70,"title":71,"news_slug":72,"published_at":73},"206ea36a-1eea-463f-a241-1e3b32f5ec2d","USTC GraphForge:证据图把任务和 rubric 钉在一起,Qwen3.6-27B 涨三基准","graphforge-ustc-qwen36-27b-evidence-graph","2026-10-04T03:05:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"8f0348fd-b4d2-4567-8dae-f2d6f9cf7760","论文实测:算术电路焊进LLM权重,6位数冲上100%","zero-gradient-arithmetic-circuits-llm-weights","2026-11-21T17:30:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"42f5ea83-365c-4a4e-a318-17d7c4c11c15","论文实测:自洽性证书在AIME上24%的题上会误触发","self-consistency-certificates-misfire-correlated-reasoning","2026-11-21T09:30:00+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"8e741f04-82a2-46da-9d30-3c31117ca70c","Ambient SFT:让预训练的自己当高温老师","ambient-sft-temperature-matching","2026-11-18T17:30:00+00:00",{"id":90,"title":91,"news_slug":92,"published_at":93},"80d2381c-82d3-4200-a481-3b859d3b3e77","余弦相似度测的同质化,其实是文本长度","embedding-cosine-length-artifact-homogenization-measure","2026-11-17T11:15:00+00:00",{"id":95,"title":96,"news_slug":97,"published_at":98},"4cdb4267-a0c3-423d-8005-504778932ae9","40份技术报告审计:73个主张只有19个有对照","llm-technical-report-ablation-audit","2026-11-17T05:30:00+00:00"]