[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-self-consistency-certificates-misfire-correlated-reasoning":3,"topics-all":38,"news-related-42f5ea83-365c-4a4e-a318-17d7c4c11c15":67},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"42f5ea83-365c-4a4e-a318-17d7c4c11c15","论文实测:自洽性证书在AIME上24%的题上会误触发","多数投票有了统计证书:用 anytime-valid 的 e-process 累积证据,靠 Ville 不等式控制 Type I 错误。arXiv 预印本指出,这套构造假定同题多次采样条件独立,真实推理流并非如此。相关性把零假设拆成两个,证书会指错对象,实测 AIME 上 24% 的题被误认证。","多数投票已经不新鲜了:同一道题让模型跑好几遍,取出现最多的那个答案,是推理评测里最省事的涨分手段。因为用得多,也有人开始给它配统计证书——用 anytime-valid 的 e-process 逐次累积证据,越过阈值就停手,靠 Ville 不等式拿到有限样本的 Type I 错误控制。这条线上的构造都假定同一道题的多次采样条件独立。一篇 arXiv 预印本(尚未同行评审,https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.26592)指出,真实推理流上这个前提不成立。\n\n## 相关性把一个零假设劈成两个\n\n论文的建模借自聚类数据的随机效应\u002F组内相关结构:每个认证单位开头抽一次潜在状态 Z,给定 Z 时各次增量独立同分布,于是边际上成为可交换、等相关的增量,相关系数记作 ρ。麻烦在于 ρ 大于 0 时,原本单一的零假设裂成两个:realized null 说的是这一次抽到的信念,population null 说的是模型的平均行为,两者只在 ρ=0 时才重合。\n\n论文强调自己并不推翻已有证书:单题、独立采样时两个假设本来就一样,结论照旧成立。问题出在相关性存在时,同一套构造控制住的是 realized null,而实际应用想要的是 population null。\n\n## 实测:相关性确实存在\n\n用 Claude Haiku 4.5(temperature 1)跑真实 rollout。GSM8K 上 300 道题各 30 次采样,组内相关系数 ρ̂=0.435,自助法 95% 区间 0.208 到 0.619;AIME 上 150 道题,ρ̂=0.480,区间 0.405 到 0.544。两条都远离 0。更有用的推论来自有效样本量:组内均值的设计效应让等效样本量趋近 1\u002Fρ,也就是 GSM8K 约 2.3、AIME 约 2.1。同一道题再多抽几次,信息量很快就见底。\n\n## 后果:证书会指错对象\n\n按论文的表述,标准下注 e-process 对 realized null 依然 anytime-valid,但换成 population null 去读,某组混合构造的渐近越界概率至少是 π=P(μ(Z)>0),未必守得住名义的 ε。\n\n真实数据上的差距正是在难的那一侧显形。AIME 上模型估计的群体 margin 是负的(θ̂=-0.15),提示多数题其实并不可靠;此时单题规则却在 24% 的题上触发越界,而名义水平是 0.05,同一批数据上的题级修正是 0.00。按难度分箱看更清楚:单题越界率在一致率 (0.5,0.7] 桶是 0.06,跨到 (0.7,0.9] 桶直接跳到 0.73,而漂移 θ 正好在这两个桶之间穿过零点。合成实验里预先埋了 ρ=0.09、π=0.37 的相关结构,单题规则在 population null 成立时仍以 0.32 的比率去认证,题级修正在 θ=0 处是 0.033。\n\n## 出路是换单位,不是加次数\n\n论文给的修复是把 e-process 挪到题级:每道题贡献一个近似独立的汇总量,population 有效性就恢复了。配套的分配分析结论有点反直觉——纯 rollout 预算下,方差最小的分法是每道题只抽一次。论文自己的说法是,单题的预算保证对 population null 是空的,再多采样也换不来关于群体的信息。\n\n## 该泼的冷水\n\n论文自己列了几条边界。题内共享前缀带来的相关性没有测(需要 tree-of-thought 或前缀共享生成);真实数据只用了 Claude Haiku 4.5 一个生产模型,论文说用开源模型(如 gpt-oss-20b 或 Qwen3-30B)复现是自然的稳健性检查,但可能改变量级甚至定性结论;任意序列相关下没有保证;Type I 错误控制不等于检验力,靠近 θ≈0 的边界可能要堆很多题才能认证。这还是一篇尚未同行评审的 arXiv 预印本。\n\n## 所以呢\n\n对做推理系统的工程含义很直接:如果流程是「同题多采 + 统计认证」,那证书可能盖的不是你以为的那个章。要拿到群体层面的保证,得把预算摊到题目维度上做题级聚合,而不是在同一道题上继续堆采样。\n\n机制层另有独立团队在同一条线上做认证:https:\u002F\u002Farxiv.org\u002Fabs\u002F2605.05873 的 CITE 针对预设目标答案做 anytime-valid 认证,https:\u002F\u002Farxiv.org\u002Fabs\u002F2510.17472 的 Certified Self-Consistency 用 Martingale Majority Certificate 做序贯停止。自洽性本身出自 ICLR 2023 的那篇论文(会议页 https:\u002F\u002Ficlr.cc\u002Fvirtual\u002F2023\u002Fposter\u002F11718),它自己报告 GSM8K 上 +17.9% 的提升——这条线的收益是真的,只是统计保证得换个单位才能算数。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.26592","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"5e628969-6d2a-437f-998a-104e4b16cfb1","ai-progress",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"40269b40-7942-4650-9672-ed2e6524d37a","ai-technology",{"id":19,"name":20,"slug":20,"description":14,"color":14},"120fa59a-ff6f-4537-9bf5-f818df636a0e","benchmark",{"id":22,"name":23,"slug":23,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"623c9793-419d-4c7e-b9da-a2ed5978f06b","en","Study: self-consistency certificates misfire on 24% of AIME","Self-consistency certificates assume independent rollouts. An arXiv preprint shows real reasoning streams violate that, splitting one null into two.","Majority voting is no longer new: run the model several times on the same question, take the answer that appears most often. It is the cheapest way to lift a reasoning score, so people started attaching statistical certificates to it, using an anytime-valid e-process that accumulates evidence and stops once it crosses a threshold, with Ville's inequality supplying finite-sample Type I error control. Every construction in that line assumes the repeated samples for one question are conditionally independent. An arXiv preprint (not yet peer reviewed, https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.26592) argues that premise fails on real reasoning streams.\n\n## Correlation splits one null into two\n\nThe modelling borrows the random-effects and intraclass-correlation structure from clustered data: each certification unit draws a latent state Z once at the start, the increments are independent and identically distributed conditional on Z, so the marginal process becomes exchangeable and equicorrelated with correlation ρ. The trouble is that once ρ is above zero, a single null splits into two. The realized null concerns the belief that was drawn this time, the population null concerns the model's average behaviour, and the two coincide only when ρ=0.\n\nThe paper stresses it does not overturn existing certificates. For a single question with independently sampled rollouts the two nulls are the same and the earlier conclusions hold as they stand. The problem appears once correlation exists: the same construction then controls the realized null, while what an application actually wants is the population null.\n\n## Measured: the correlation is there\n\nThe runs use Claude Haiku 4.5 at temperature 1. On GSM8K, 300 prompts with 30 rollouts each, the intraclass correlation is ρ̂=0.435 with a bootstrap 95% interval from 0.208 to 0.619; on AIME, 150 problems, ρ̂=0.480 with an interval from 0.405 to 0.544. Both sit well away from zero. The more useful consequence follows from effective sample size: the design effect on the within-unit mean pushes the variance-equivalent sample size toward 1\u002Fρ, which is about 2.3 on GSM8K and 2.1 on AIME. Drawing more rollouts for the same question stops buying information quickly.\n\n## What goes wrong: the certificate points at the wrong object\n\nIn the paper's own terms, the standard betting e-process stays anytime-valid for the realized null, but read as a population-null certificate, a suitable mixture crosses with asymptotic probability at least π=P(μ(Z)>0) and need not respect the nominal ε.\n\nThe gap shows up on the harder side of the real data. On AIME the estimated population margin is negative (θ̂=-0.15), suggesting the model is unreliable on most questions, and yet the single-unit rule crosses on 24% of prompts while the nominal level is 0.05, whereas the prompt-level remedy declines on the same data at a rate of 0.00. Binning by difficulty makes it clearer: the single-unit crossing rate is 0.06 in the (0.5,0.7] agreement bin and jumps to 0.73 in the (0.7,0.9] bin, with the drift θ crossing zero between exactly those two bins. In the synthetic study, where a correlated structure with ρ=0.09 and π=0.37 is planted, the single-unit rule still certifies the population null at a rate of 0.32, while the prompt-level remedy sits at 0.033 when θ=0.\n\n## The fix changes the unit, not the count\n\nThe remedy is to move the e-process up to the prompt level, where each question contributes a nearly independent summary and population validity is restored. The accompanying allocation result runs against intuition: under a pure rollout budget, the variance-minimising split is one rollout per prompt. In the paper's phrasing, the single-unit budget guarantee is vacuous for the population null, and sampling more cannot buy information about the population.\n\n## Where to push back\n\nThe paper lists its own limits. The correlation coming from a shared prefix within a question is not measured, and measuring it would need tree-of-thought or prefix-shared generation. The real data uses a single production model, Claude Haiku 4.5, and the paper says a reproducible open-model check on something like gpt-oss-20b or Qwen3-30B is the natural robustness test but could shift the magnitudes or even the qualitative pattern. There is no guarantee under arbitrary serial dependence. Type I error control is not power, so near the θ≈0 boundary certification may need many units. And this is still an arXiv preprint that has not been peer reviewed.\n\n## So what\n\nFor anyone building a reasoning system the implication is direct: if the pipeline is 'many rollouts per question plus a statistical certificate', the certificate may not be stamping what you think it is. Getting a population-level guarantee means spreading the budget across questions and aggregating at the prompt level, rather than piling more samples onto one question.\n\nOther independent teams work on the certification side: CITE (https:\u002F\u002Farxiv.org\u002Fabs\u002F2605.05873) targets anytime-valid certification of a prespecified target answer, and Certified Self-Consistency (https:\u002F\u002Farxiv.org\u002Fabs\u002F2510.17472) uses a Martingale Majority Certificate for sequential stopping. Self-consistency itself comes from the ICLR 2023 paper (conference page https:\u002F\u002Ficlr.cc\u002Fvirtual\u002F2023\u002Fposter\u002F11718), which reports a +17.9% gain on GSM8K. The gains on this line are real; the statistical guarantee only counts once it moves to the right unit.","self-consistency-certificates-misfire-correlated-reasoning","2026-11-21T09:30:00Z","2026-11-21T09:15:05.639099Z","2026-11-21T09:15:05.639110Z",true,"agent",29,[39,49,58],{"slug":40,"tag_slug":41,"title_zh":42,"title_en":43,"intro_zh":44,"intro_en":45,"id":46,"is_active":35,"created_at":47,"modified_at":48},"image-generation","text-to-image","图像生成：从文生图到逐框编辑的模型竞赛","Image Generation: from text-to-image to frame-level editing","文生图已是红海，战场正转向编辑与可控性：FLUX.2\u002F3 用开源和量化把生图搬进消费级显卡，Qwen-Image、Seedream 等国产模型紧咬不放，GPT Image 2 则把推理能力带进像素。本专题追踪图像生成与编辑模型的发布、架构演进、加速量化与商业化数据，看「好图」如何变成「可用的图」。","Text-to-image is a crowded race, and the frontier has moved to editing and control: FLUX.2\u002F3 push open weights and quantization onto consumer GPUs, Qwen-Image and Seedream close in from China, and GPT Image 2 brings reasoning into pixels. This topic tracks model releases, architecture shifts, acceleration work, and the business numbers behind \"pretty\" becoming \"usable\".","52de7c59-7864-4569-8ae7-3d656d26a1a1","2026-11-20T07:20:23.534506Z","2026-11-20T07:20:23.534527Z",{"slug":50,"tag_slug":50,"title_zh":51,"title_en":52,"intro_zh":53,"intro_en":54,"id":55,"is_active":35,"created_at":56,"modified_at":57},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":59,"tag_slug":59,"title_zh":60,"title_en":61,"intro_zh":62,"intro_en":63,"id":64,"is_active":35,"created_at":65,"modified_at":66},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":68},[69,74,79,84,89,94],{"id":70,"title":71,"news_slug":72,"published_at":73},"67340a61-0b6f-4fac-b8a1-e430d206634e","9.3%涨到32.0%:角色向量读得出,替不了提示词","system-prompt-role-vectors-llm-steering","2026-11-13T11:30:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"78b52428-2dbb-4819-aa2a-eee5bccf7d5a","用大模型筛基准:基准反而变简单了","llm-validation-minimal-pair-benchmarks-easier","2026-11-11T11:20:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"e5c12c0b-5f79-47a4-becb-47d4e0c5d157","126万次盲投:模型没变,是人在变挑剔","chatbot-arena-evaluation-drift-pickier-judges","2026-11-11T07:05:00+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"409b5265-425f-4a26-a4a4-4a232f1cc1e0","自动指标看不见的失败:4个模型翻车实录","romanized-indic-generation-metric-blindness","2026-11-09T09:05:00+00:00",{"id":90,"title":91,"news_slug":92,"published_at":93},"26602719-1a86-4264-812f-a57a67b966b7","1911大英百科对儿童语料:小模型排名反转","small-model-corpus-choice-skill-profile-reversal","2026-11-09T01:12:32+00:00",{"id":95,"title":96,"news_slug":97,"published_at":98},"1b51020b-9a0c-46f8-98ec-e4e0c8c051a6","基准分其实是难度的加权平均:25 个主流基准的体检报告","benchmark-difficulty-measure-coverage-gaps","2026-11-01T01:16:42+00:00"]