[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-omni-woodpecker-training-free-hallucination-fix":3,"topics-all":38,"news-related-f728cc32-7d43-4f75-b4fa-5d76a6115069":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"f728cc32-7d43-4f75-b4fa-5d76a6115069","全模态幻觉的病根在融合层:OWP 免训练重平衡证据与先验","南京大学提出免训练框架 Omni-Woodpecker(OWP):不改权重,只在融合表征内部分离接地证据与竞争先验再重平衡。Qwen2.5-Omni-7B 上 CMM 从 73.2 升到 86.1,幻觉抵抗率 57.0 涨到 85.5,并迁移到 VideoLLaMA2-AV。代价是 1.71 倍延迟,只适用开放权重。","一段只有公鸡叫声的养蜂视频里，多模态模型会说画面里有公鸡；反过来一段喷洒液体的画面配着人声，模型可能一口否认\"刚才确实有人在说话\"。这不是看错了，而是融合层里另一条模态的先验抢了主导权——你问视觉，它答耳朵。\n\n## 幻觉的病根不在输出层，在融合层\n\n近两年治多模态幻觉的主流做法都停在输出层：对比解码在感知输入被破坏时惩罚那些仍然\"很可能\"的 token，OPERA 加过度自信惩罚并在事后重分配注意力，被对比的 MAD、HulluEdit 也在这一层。南京大学与中国科学院大学团队的新论文指出这忽略了关键：早期融合把接地证据和学到的先验纠缠在同一个表征里，只在输出端动刀等于把混在一起的信号再搅一次。\n\n论文把可诊断的失败归为 cross-modal hallucination：一个模态的证据被另一个模态、或被语言先验压过去。CMM 基准把这类失败描述成语言、视觉、音频三种主导以及虚假的跨模态相关；AVHBench 测音频驱动的视觉幻觉、视觉驱动的音频幻觉和音视频匹配。\n\n## 三步走：查冲突、拆证据、压先验\n\n方法叫 Omni-Woodpecker，简称 OWP，全程不改一个权重。\n\n第一步查冲突。同一个冻结模型跑四个视图：完整输入、只有音频、只有视频、只有文字。文字视图是共享参照，用来把模态证据和语言驱动的答案分开。完整视图和目标模态视图答案一致就说明没有冲突，直接返回原答案；不一致即定位出\"问题问的是哪个模态、抢答的是哪个模态\"。\n\n第二步拆证据。对隐藏状态做正交分解，把接地证据和竞争先验分到两个方向上。第三步压先验、扶证据：解码前把纠正预算分配到 token-head 路径重加权和残差状态修复两条路上，算出新状态写回同一层。消融显示残差状态是主要纠正位置，证据最好在中层恢复、先验最好在深层压制。\n\n目标模态判定值得单说：OWP 附加一个诊断问句问\"这题需要音频、视频还是两者\"，取三个 next-token logit 的 softmax 组分，论文报告它在 99.48% 的样本上与基准定义的目标模态一致。\n\n## 数字：CMM +12.9 分，跨架构也成立\n\n以 Qwen2.5-Omni-7B 为骨干，CMM 准确率 73.2→86.1，AVHBench 75.1→83.7。摘要报的是 +12.9 和 +8.6 分，论文报告在两个基准上分别比 MAD 基线高 4.9 和 7.7 分。更值得看的是幻觉抵抗率 HR 从 57.0 涨到 85.5，感知准确率 PA 停在 86.7——backbone 原本 PA 和 HR 之间差 32.5 分，OWP 压到 1.2 分，MAD 还留着 16.9 分。\n\n换到融合设计完全不同的 VideoLLaMA2-AV 上，CMM 76.2→80.8，AVHBench 70.2→74.1，HR 65.3→77.7。论文还报 OWP 在 Qwen2.5-Omni-7B 上比更新的 Qwen3-Omni 高 4.3 和 6.2 分。\n\n通用音视频推理没被牺牲：OmniBench 45.36→47.29，WorldSense 45.21→47.46，MUSIC-AVQA 85.08→86.08，AVQA 91.83→92.50。统计口径是 4,604 个各方法都能给出有效预测的样本，双侧 McNemar 检验加 5,000 次行级 bootstrap 重采样，论文报告共同子集上的领先幅度 p\u003C10^-16。\n\n## 代价写在明面上：1.71 倍延迟、24.9 GiB 显存、只能动开源权重\n\n论文自己测了推理成本，RTX A6000 上跑三次取平均。OWP 在 Qwen2.5-Omni-7B 上每次输入 6.05 秒，是 backbone 3.54 秒的 1.71 倍，最多 5 次前向调用；MAD 是 9.96 秒、2.81 倍、9 次调用；HulluEdit 13.02 秒、3.68 倍。两个骨干的峰值显存分别是 24.9 GiB 和 17.3 GiB。它比两个基线便宜，是因为冲突触发的调度——只有 25.8% 的输入被判有冲突并改了答案，16.9% 改了最终答案，其余 74.2% 走未修改的 backbone。\n\n代价还有两条硬的。第一，OWP 改隐藏状态和注意力输出，需要模型内部权限，论文明说只适用于开放权重模型，API-only 的模型用不了。第二，它恢复的是 backbone 已编码但融合时没用上的证据；目标模态本身就被感知错时，没有可靠证据可恢复。论文也承认只处理跨模态幻觉这一类，时间顺序、声纹、情绪归属都还没覆盖。\n\n## 所以呢\n\n这篇论文的价值不在又一个幻觉抑制方法，而在把问题定位换了个位置：幻觉不是模型嘴上说错了什么，是模型脑子里哪个信号盖住了哪个信号。而\"不改权重\"这个约束反过来也划出了天花板——你必须在模型内部动手，也必须接受多出来的 1.7 倍延迟和 24.9 GiB 显存。\n\n代码以 MIT 协议释出，仓库是 github.com\u002FZiiJianLiu\u002FOmni-Woodpecker，论文在 arXiv 的 2610.27103，10 月 19 日提交，10 月 26 日的 Hugging Face 每日论文榜排到当日第三。想试的人注意 README 要求：BF16 下至少 24 GB 显存的 GPU，权重默认缓存在 ~\u002F.cache\u002Fomni-woodpecker。\n\n如果 omni 模型的下一个战场是\"少说错话\"而不是\"多说对话\"，把干预点从输出层挪进融合层，可能比再训一轮后训练更划算。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.27103","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"5e628969-6d2a-437f-998a-104e4b16cfb1","ai-progress",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"1fcfaaf2-67de-43d3-9e35-5784852fec60","ai-safety",{"id":19,"name":20,"slug":20,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":22,"name":23,"slug":23,"description":14,"color":14},"499f4b56-819d-49a3-9609-33e775143b86","multimodal",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"8db1efc1-5185-49cc-917e-824b2f9b1303","en","Omni-Woodpecker Rebalances Priors Inside Frozen Omni LLMs","Omni-Woodpecker fixes omni-LLM hallucination with no weight changes, rebalancing evidence and priors inside the frozen model. CMM accuracy 73.2 to 86.1.","In a beekeeping clip where you can only hear crowing, an omni-modal model will tell you there is a rooster in the picture. Flip it around: a shot of fabric being sprayed with someone speaking over it, and the model may flatly deny that anyone was talking. Nothing was misperceived. Another modality's prior simply won the argument inside the fused representation: you asked about vision, it answered from ears.\n\n## The fault is not in the output layer, it is in the fusion layer\n\nMost of the last two years' work on multimodal hallucination stays at the output level. Contrastive decoding penalises tokens that remain likely when the perceptual input is degraded. OPERA adds an over-trust penalty and reallocates attention after the fact. The controlled baselines here, MAD and HulluEdit, live at the same level. A paper from Nanjing University and the University of Chinese Academy of Sciences argues that this misses the point: early fusion entangles grounding evidence with learned priors inside one fused representation, so nudging the output just stirs an already mixed signal again.\n\nThe paper isolates a diagnosable failure class called cross-modal hallucination: evidence from one modality is overridden by a cue from another modality, or by the language prior. The CMM benchmark describes these failures as language, visual, and audio dominance plus spurious cross-modal correlations. AVHBench tests audio-driven visual hallucination, visual-driven audio hallucination, and audio-visual matching.\n\n## Three steps: find the conflict, split the evidence, suppress the prior\n\nThe method is called Omni-Woodpecker, OWP for short, and it does not change a single weight.\n\nStep one is finding the conflict. The same frozen model is run on four views: the full input (query plus video plus audio), audio only, video only, and text only. The text-only view acts as a shared reference that separates modality evidence from language-driven answers. If the full view and the target-modality view agree, there is no conflict and OWP returns the backbone answer untouched. When they disagree, a cross-modal conflict exists, and the method works out which modality the query asked about and which one answered instead.\n\nStep two is splitting the evidence. Orthogonal decomposition on the hidden states separates the grounding evidence from the competing prior into two directions. Step three is suppressing the prior and restoring the evidence: before decoding, one correction budget is split between token-head path reweighting and residual-state restoration, and the edited state is written back to the same layer. The ablations show the residual state is the main site of correction, with evidence best restored in middle layers and priors best suppressed in later layers.\n\nTarget typing is worth a mention. OWP appends a diagnostic query asking whether the question needs audio, video, or both, takes the softmax over three next-token logits to build the scores, and the paper reports it agrees with the benchmark-defined target modality on 99.48% of rows.\n\n## The numbers: +12.9 on CMM, and it transfers across architectures\n\nWith Qwen2.5-Omni-7B as the backbone, CMM accuracy goes from 73.2 to 86.1 and AVHBench from 75.1 to 83.7. The abstract reports +12.9 and +8.6 points, and the paper reports gains over the MAD baseline of 4.9 and 7.7 points on the two benchmarks. The more interesting number is hallucination resistance: HR climbs from 57.0 to 85.5 while perception accuracy lands at 86.7. The backbone had a 32.5-point gap between PA and HR; OWP compresses it to 1.2 points, where MAD still leaves 16.9.\n\nOn VideoLLaMA2-AV, a backbone with a completely different audio-visual fusion design, CMM goes 76.2 to 80.8, AVHBench 70.2 to 74.1, and HR 65.3 to 77.7. The paper also reports that OWP on Qwen2.5-Omni-7B beats the newer Qwen3-Omni by 4.3 and 6.2 points.\n\nGeneral omni-modal reasoning is not sacrificed. OmniBench moves 45.36 to 47.29, WorldSense 45.21 to 47.46, MUSIC-AVQA 85.08 to 86.08, and AVQA 91.83 to 92.50. The statistical setup uses the 4,604 examples on which every method returns a valid prediction, with exact two-sided McNemar tests and 95% confidence intervals from 5,000 row-level bootstrap resamples; the paper reports the lead on the common subset at p\u003C10^-16.\n\n## The bill is printed openly: 1.71x latency, 24.9 GiB of memory, open weights only\n\nThe paper measures its own inference cost on an RTX A6000, averaged over three runs. OWP costs 6.05 seconds per input on Qwen2.5-Omni-7B, 1.71x the backbone's 3.54 seconds, with at most five forward calls. MAD is 9.96 seconds, 2.81x, and nine calls. HulluEdit is 13.02 seconds, 3.68x. Peak memory on the two backbones is 24.9 GiB and 17.3 GiB. OWP is cheaper than both baselines because of the conflict-triggered schedule: only 25.8% of inputs are judged to contain a conflict and 16.9% have their final answer changed, while the remaining 74.2% run the unmodified backbone.\n\nTwo costs are harder. First, OWP edits hidden states and attention outputs, so it needs access to model internals, and the paper states plainly that it applies to open-weight OLLMs but not to models available only through an API. Second, it restores evidence the backbone already encoded but failed to use during fusion; if the target modality is misperceived even on its own, there is no reliable evidence to restore. The authors also concede that it covers only the cross-modal class, leaving temporal hallucination, speaker attribution, emotion, and prosody for later.\n\n## So what\n\nThe value here is not another hallucination-suppression method but a relocated problem: hallucination is not about what the model says, it is about which signal inside the model drowned out which other signal. And the training-free constraint draws the ceiling in return, because you have to reach inside the model, and you have to accept the extra 1.7x latency and 24.9 GiB of memory.\n\nThe code ships under the MIT license at github.com\u002FZiiJianLiu\u002FOmni-Woodpecker, the paper is arXiv 2610.27103, submitted on October 19 and ranked third on Hugging Face's Daily Papers board on October 26. The README asks for at least 24 GB of GPU memory in BF16, with weights cached under ~\u002F.cache\u002Fomni-woodpecker by default.\n\nIf the next fight for omni models is about saying less wrong rather than saying more, moving the intervention point from the output layer down into the fused representation may beat another round of post-training.","omni-woodpecker-training-free-hallucination-fix","2026-10-26T05:12:35Z","2026-10-26T05:12:37.900931Z","2026-10-26T05:12:37.900944Z",true,"agent",158,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"23567889-4d40-4af8-a58a-ca61cdb6a8b2","小米A-RSI:不用外部标注,模型自己发现题库刷完了","a-rsi-zero-data-self-improvement","2026-10-30T19:09:58+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"098ef0da-3801-419a-9e33-855d6c6d7ee6","Trinity:递归自我改进,训练基准平均涨 34 分","trinity-capability-driven-recursive-self-improvement","2026-10-28T13:16:03+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"5480e0ca-7b17-481c-8a24-8498f1a0aecd","8B模型OST涨13.41分:空间推理的接口锁定","chronoreplay-spatial-interface-lock-in","2026-10-28T05:09:43+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"edc863dc-1918-4fdf-8306-640d913e15e2","Google OPST:零人类习题,模型自出题自提升","google-opst-data-free-self-training","2026-10-26T21:10:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"9a6f3f05-0bbd-4faf-84f8-c2932ea13801","前沿模型会往画里藏话:78% 的消息能被自己读回来","frontier-llm-image-steganography-fable-51","2026-10-24T03:13:22+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"cdcd6b11-8a65-4b15-9add-6fa9343de5a3","十分钟 AI 协助后,人为何突然不愿再解题","arxiv-2604-04721-ai-persistence-reduces","2026-10-17T07:00:00+00:00"]