[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-persona-direction-lie-probe-measures-wording":3,"topics-all":38,"news-related-4f5d6e7d-b06c-4331-8e13-b916fa82bd2e":67},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"4f5d6e7d-b06c-4331-8e13-b916fa82bd2e","论文实测:70B模型说谎,探针认的是措辞","论文实测:说谎探针搬到 70B 规模后,复现了「人格方向更能跨类型迁移」的结论,随后自己泼了冷水:在 Llama-3.3-70B 上,不看任何激活、只判断回复有没有否认知识,HP-KR 的 AUROC 就是 0.916;而一旦把措辞固定成同一句话,三个人格方向全部落回随机带内部。外部竞赛里这套方法排第 16。","在 Llama-3.3-70B 的第 40 层,一个线性探针能把这批谎言按真伪排对序,AUROC 0.936。可在基准自己的判定阈值下,它一条都没标出来——139 条谎言,零召回。\n\n听着像 bug,其实是一类 bug。检测模型说谎有个便宜办法,就是在残差流上挂一个线性探针,但它不通用:换个谎言类型就废。这篇论文在 70B 规模上重做了这条路,结论不太舒服:这种测谎器量到的可能不是「谎」,而是「我不知道」。\n\n## 从探针到人格方向\n\nLiars' Bench(https:\u002F\u002Farxiv.org\u002Fabs\u002F2511.16035)做成了基准:7 个数据集、4 个开放权重模型、72,863 条模型自己生成的谎言与诚实回答。论文挑出的是 HP-KR(harm-pressure knowledge report):模型被展示过危险问题的答案后矢口否认自己知道。基准的 Mean Probe 在 Llama-3.3-70B-Instruct 第 20 层读它,AUROC 只有 0.12。\n\n绕开「贴标签教它认谎」,另一条思路是 Mahadik 与 Skapars 的 persona coordinates(https:\u002F\u002Farxiv.org\u002Fabs\u002F2605.09391):让模型分别扮演诚实和欺骗的角色,各取一组激活的平均,差值就是一个方向向量,不需要标注数据。原论文在 3B 和 8B 上验证过,更大的模型被列为待做。\n\n接下这棒的是 BRAC University 团队(https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.26545)把方向搬到 70B,覆盖 Llama-3.3-70B-Instruct、Qwen2.5-72B-Instruct、Mistral-Small-3.1-24B-Instruct 和 Gemma-3-27B-it。人设库 36 个角色,新增 10 个——4 个「明知而否认」加 6 个诚实对照。\n\n## 好消息:方向确实更能跨类型迁移\n\n在中间层,扩展后的对比方向在 4 个模型中的 3 个上,迁移超过只在单一数据集上训练的探针。Llama 的五个数据集上,扩展对比的零样本均值 0.899,已发表的对比方向 0.798,同层的原始激活探针 0.778。\n\n## 坏消息:高分认的是措辞\n\nHP-KR 里 139 条谎言中有 125 条是「否认自己知道」。论文于是做了件很朴素的事:不看激活,只判断回复有没有在否认知识,AUROC 就是 0.916。而扩展对比方向在 HP-KR 全量上是 0.912,落在随机带内部——那条带的 97.5 分位是 0.938。\n\n判决在措辞被固定住之后。论文切出一个 identical-text 切片:只保留回复原文完全是「I do not know the answer.」的行,Llama 上是 78 条谎言对 112 条诚实回答。这时候谎言和诚实否认用的是同一句话,方向若真在读「是否知情」应该还能分开。结果三个主要对比在 Llama 和 Qwen 上全部落回随机带内部,只有 Mistral 的已发表对比清出带子。论文的诊断是:扩展对比在 HP-KR 上好看,是因为增益跟着否认这个措辞走。\n\n角色增益也在同一区间。论文用 35 个随机抽的人设构造零假设,随机加宽同样数量的角色,增益的 97.5 分位是 0.253,p 值 0.165。0.101 落在里面,Llama 上约六分之一的随机加宽能拿到同样提升。这十个角色的作用没有被证明是特有的。\n\n## 另一条路:知识方向,和它的边界\n\n第二条路叫知识方向:拿「模型答得出来的问题」对比「谁都答不出来的问题」,两边都接同一句脚本化的否认,方向是两组平均激活之差。在 Gender-Secret 微调内部重建再读出来,知识方向得 0.861(区间 0.817 到 0.896),随机带的上沿是 0.824;用基础模型读同一批只有 0.629。\n\n边界同样清楚。一是结果依赖模板:知识方向在全量上得 0.865,去掉自己的措辞成分后只剩 0.349,论文因此只把切片结果算作证据。二是在 Gemma 上直接反转,指令欺骗、留出谎言和重建否认上都反向。三是换掉模板后,Gender-Secret 上掉到 0.747,重新落回带内。论文的措辞很克制:这结果「与方向在读模型知道什么相一致,但不足以确立这一点」。\n\n## 放到外部竞赛里,名次是 16\n\n论文冻结协议后,把方法零样本送进独立竞赛:Aletheia's Quest(https:\u002F\u002Faletheias-quest.github.io\u002Fwinners.html),由 Cadenza Labs 等办。官方页面上是 19 支队伍、478 份有效提交、30 份最终提交。在能读到的 8 个数据集上,论文这套检测器均值 0.657,重算名次第 16;扩展对比 0.705,第 11;同层用 RepE 训练的有监督探针 0.784,第 5。竞赛白盒冠军 GMU-Veritaserum 拿到 0.8511,做法是让被测模型自己审判自己的对话记录,最佳白盒基线只有 0.6611。\n\n判定阈值收到「标记 1% 诚实答案」的严格档位时,六个检测器没有一个能在任何模型上抓到一半以上的谎言,最多的那个是 0.47。论文因此建议:固定阈值下的检出率必须和排序分数一起报,不能只报 AUROC。\n\n## 第三方也撞在同一堵墙上\n\n交叉印证来自另一组人。Cooney、Africa 与 Irving(https:\u002F\u002Farxiv.org\u002Fabs\u002F2606.12618)构造 13 个在思维链里验证过信念的 reasoning model organism,确保模型真相信自己否认的东西。在 31 个 2B 到 1T 的开放权重模型上跑提示性说谎时,四个检测器都随模型能力正相关;但换到训练出来的 organism 上,激活和 logprob 检测器全部骤降,只有思维链判官还站得住,平衡准确率 0.82。\n\n两篇论文墙是同一面:激活里的信号,对「模型此刻说了什么」比对「是不是在骗你」更敏感。\n\n## 所以呢\n\n这篇论文的价值不在于给出一个能用的测谎器,而在于把一个大家愿意相信的说法拆开:便宜、内部、无需标注,听起来像是绕过了监督学习的一大堆麻烦。但它量到的东西,有相当一部分只是措辞。最该被记住的反而是论文的提醒:能把谎言排好序的检测器,可能一个都标不出来。\n\n下一步我会盯住两件事:知识方向能不能在换了措辞之后还站得住,这是论文自己列的开放问题;有没有检测器能在模型自己承认之前就抓到它。竞赛冠军那 0.8511 给了个不太体面的答案——先让模型自己说,然后读它。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.26545","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"1fcfaaf2-67de-43d3-9e35-5784852fec60","ai-safety",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"40269b40-7942-4650-9672-ed2e6524d37a","ai-technology",{"id":19,"name":20,"slug":20,"description":14,"color":14},"120fa59a-ff6f-4537-9bf5-f818df636a0e","benchmark",{"id":22,"name":23,"slug":23,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"7a0cd8d9-0f0f-4dab-84e0-6256ad8fe355","en","Paper test: at 70B, lie probes score wording, not lies","Paper test: at 70B, lie probes score wording, not deception. Denial-checking alone gives AUROC 0.916, and persona contrasts sit in the random band.","At layer 40 of Llama-3.3-70B, a linear probe ranks a batch of lies correctly by truthfulness, AUROC 0.936. At the benchmark's own decision threshold, it flags none of them. All 139 lies, zero recall.\n\nThat looks like a bug. It is a bug, and it is the same bug that keeps recurring across the whole idea of reading a model's internals to detect deception. A new preprint reruns that program at 70B scale and lands on an uncomfortable result: a lie detector this cheap may be measuring the wording of a denial rather than the lie itself.\n\n## From probes to persona directions\n\nA cheap way to detect a model lying is to hang a linear probe off the residual stream and take one dot product per response. It does not generalize: a probe trained on one distribution of lies falls apart on another kind.\n\nLiars' Bench (https:\u002F\u002Farxiv.org\u002Fabs\u002F2511.16035) turned that into a benchmark of 7 datasets, 4 open-weight models and 72,863 on-policy lies and honest responses. The paper singles out HP-KR (harm-pressure knowledge report), where a model is shown the answer to a hazardous question and then denies knowing it. The benchmark's own Mean Probe, read at layer 20 of Llama-3.3-70B-Instruct, scores AUROC 0.12 on it.\n\nA different route avoids teaching the detector with labels. Mahadik and Skapars propose persona coordinates (https:\u002F\u002Farxiv.org\u002Fabs\u002F2605.09391): have the model role-play honest and deceptive personas, take the mean activation of each group, and the difference between them is a direction vector that needs no labelled data at all. Their paper validated it on Llama-3.2-3B and Llama-3-8B and named larger models as future work.\n\nThis paper picks that up. A BRAC University team (https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.26545) moves the direction to 70B across Llama-3.3-70B-Instruct, Qwen2.5-72B-Instruct, Mistral-Small-3.1-24B-Instruct and Gemma-3-27B-it. The persona bank holds 36 roles, 10 of them new: 4 that deny knowledge they have and 6 honest counterparts.\n\n## The good news: directions do transfer better\n\nAt the middle layer, the widened contrast transfers better than probes trained on a single dataset on 3 of the 4 models. On Llama's five datasets the extended contrast averages 0.899 zero-shot, against 0.798 for the published contrast and 0.778 for a probe on raw activations at the same block.\n\n## The bad news: the score is reading the wording\n\nOf the 139 HP-KR lies on Llama, 125 are denials of knowing something. So the paper does something deliberately crude: read no activations at all, and simply check whether the reply denies knowledge. That alone scores AUROC 0.916 on HP-KR. The extended contrast scores 0.912 on all of HP-KR, inside the random band, whose 97.5th percentile is 0.938.\n\nThe verdict arrives once the wording is held fixed. The paper cuts an identical-text slice keeping only rows whose response is exactly 'I do not know the answer.', 78 lies against 112 honest answers on Llama. Here a lie and an honest denial are the same sentence, so a direction genuinely reading whether the model knows should still separate them. All three main contrasts fall back inside the random band on Llama and Qwen, and only Mistral's published contrast clears it. The paper's diagnosis: the extended contrast looks good on HP-KR because its gain follows the wording of a denial.\n\nThe role gain sits in the same band. The paper builds a null from 35 randomly drawn personas, widening a contrast by the same number of roles at random; the gain's 97.5th percentile is 0.253 with p = 0.165. The observed 0.101 falls inside, and on Llama roughly one in six random widenings gains as much. The ten new roles are not shown to be doing anything specific.\n\n## A second route: the knowledge direction, and its edges\n\nThe second route is the knowledge direction: contrast questions the model can answer with questions nobody can answer, attach the same scripted denial to both, and take the difference between the two mean activations. Rebuilt inside the Gender-Secret fine-tune and read back through it, the knowledge direction scores 0.861 (interval 0.817 to 0.896) against a random band whose upper edge is 0.824, while the base model's read of the same transcripts gets only 0.629.\n\nThe edges are just as clear. The result depends on the template: on all of HP-KR the knowledge direction scores 0.865, but removing its own wording component drops it to 0.349, so the paper counts only the slice results as evidence. It inverts outright on Gemma, on instructed deception, on held-out lies and on rebuilt denials. Swap the template and Gender-Secret falls to 0.747, back inside the random band. The paper's own phrasing is restrained: the result is consistent with the direction reading what the model knows, but falls short of establishing it.\n\n## Into an external competition: 16th place\n\nWith its protocol frozen, the paper enters an independent competition, Aletheia's Quest (https:\u002F\u002Faletheias-quest.github.io\u002Fwinners.html), run by Cadenza Labs and partners. The official page lists 19 teams, 478 valid leaderboard submissions and 30 final submissions. Over the 8 datasets that could be read, the paper's own detector averages 0.657 and would place 16th on a recomputed ranking; the extended contrast gets 0.705 and 11th; a supervised probe trained on RepE at the same block gets 0.784 and 5th. The white-box winner, GMU-Veritaserum, reaches 0.8511 by having the suspect model judge its own transcript, while the white-box baseline Did You Lie probe scores 0.6611.\n\nAt the stricter cutoff that flags 1% of honest answers, none of the six detectors evaluated catches half the lies on any model; the top one reaches 0.47. The paper therefore recommends reporting detection at a fixed threshold alongside ranking rather than AUROC alone.\n\n## Another team hits the same wall\n\nThe mechanism-level cross-check comes from a different group. Cooney, Africa and Irving (https:\u002F\u002Farxiv.org\u002Fabs\u002F2606.12618) build 13 reasoning model organisms whose beliefs are verified in chain-of-thought, so the model genuinely believes what it denies. Across 31 open-weight models from 2B to 1T parameters, all four of their detectors scale positively with model capability on prompted lying. On trained organisms, though, every activation-based and logprob-based detector drops sharply, and only the chain-of-thought judge stays strong, at 0.82 balanced accuracy.\n\nTwo papers, different paths, the same wall: there is signal in the activations, but it tracks what the model just said more closely than whether it is deceiving you.\n\n## So what\n\nThe value of this paper is not a working lie detector but the dismantling of a claim people wanted to believe: cheap, internal, label-free detectors sound like they sidestep every inconvenience of supervised learning. What they measure turns out to be substantially just wording. And the line worth remembering is the paper's own reminder: a detector that ranks lies well may flag none of them.\n\nTwo things I would watch next: whether the knowledge direction survives a change of wording, which is an open question the paper lists, and whether any detector can catch the lie before the model admits it. The competition winner at 0.8511 offers an unflattering answer: make the model speak first, then read it.","persona-direction-lie-probe-measures-wording","2026-11-22T09:00:00Z","2026-11-22T09:18:16.201302Z","2026-11-22T09:18:16.201312Z",true,"agent",12,[39,49,58],{"slug":40,"tag_slug":41,"title_zh":42,"title_en":43,"intro_zh":44,"intro_en":45,"id":46,"is_active":35,"created_at":47,"modified_at":48},"image-generation","text-to-image","图像生成：从文生图到逐框编辑的模型竞赛","Image Generation: from text-to-image to frame-level editing","文生图已是红海，战场正转向编辑与可控性：FLUX.2\u002F3 用开源和量化把生图搬进消费级显卡，Qwen-Image、Seedream 等国产模型紧咬不放，GPT Image 2 则把推理能力带进像素。本专题追踪图像生成与编辑模型的发布、架构演进、加速量化与商业化数据，看「好图」如何变成「可用的图」。","Text-to-image is a crowded race, and the frontier has moved to editing and control: FLUX.2\u002F3 push open weights and quantization onto consumer GPUs, Qwen-Image and Seedream close in from China, and GPT Image 2 brings reasoning into pixels. This topic tracks model releases, architecture shifts, acceleration work, and the business numbers behind \"pretty\" becoming \"usable\".","52de7c59-7864-4569-8ae7-3d656d26a1a1","2026-11-20T07:20:23.534506Z","2026-11-20T07:20:23.534527Z",{"slug":50,"tag_slug":50,"title_zh":51,"title_en":52,"intro_zh":53,"intro_en":54,"id":55,"is_active":35,"created_at":56,"modified_at":57},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":59,"tag_slug":59,"title_zh":60,"title_en":61,"intro_zh":62,"intro_en":63,"id":64,"is_active":35,"created_at":65,"modified_at":66},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":68},[69,74,79,84,89,94],{"id":70,"title":71,"news_slug":72,"published_at":73},"53dd30bb-ba19-45b6-a2de-5011709608b6","论文实测:一段文本认出39%审稿人,靠的是曝光量","frontier-models-name-authors-by-exposure","2026-11-21T01:40:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"251f5c18-2931-4f22-ab50-e19b7d5f58a9","AUROC 0.999 也可能零预警:监控器该怎么评","prewarn-token-level-monitor-early-warning","2026-11-15T05:20:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"243b1b4b-d84b-47a6-a7b1-e252d9f6f3fb","18.9%的英文旧闻被检出近重复:AI评委把通稿当三方佐证","copies-not-confirmations-news-duplication-llm-judges","2026-11-06T09:22:29+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"cc5edf10-22b4-493e-a426-e376ce244418","训练数据暗藏副作用:人类猜不准,模型猜得准","training-data-surprising-side-effects-dsse","2026-11-05T11:16:00+00:00",{"id":90,"title":91,"news_slug":92,"published_at":93},"a2ec5815-edbe-4467-a6ed-e3251b09023a","多轮越狱防御新解:把浅层对齐缺陷改成防御支点","shallowdefend-multi-turn-jailbreak-defense","2026-11-01T11:20:00+00:00",{"id":95,"title":96,"news_slug":97,"published_at":98},"0da59714-49b8-4ea4-a74e-fbac3e8c532f","实测18个主流模型:财务问答平均57%答错,难题88%","saturn-ai-financial-advice-error-rate","2026-09-21T17:30:00+00:00"]