[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-post-training-changes-readout-not-representations":3,"topics-all":38,"news-related-f9cabba1-6f17-4aef-8a62-2bb865468313":67},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"f9cabba1-6f17-4aef-8a62-2bb865468313","论文实测:后训练改变模型读法,知识没变","波茨坦大学单作者研究对比 6 个开源底座与其后训练版:线性探针能把德语 AI 文本与人写文本分开(中位 d′ 达 3.70),答案却接近随机甚至反向;后训练平均削掉答案灵敏度 0.34,变化发生在 SFT 阶段、由早期层携带。否定句案例方向相反,SFT 修复了底座的误读,训练研究显示 2000 条标签即可定读法方向。","把 AI 写的德语文本和人写的混在一起,问模型「这段是谁写的」——模型内部其实分得清,答案却常常反着判。波茨坦大学 Jochen Laubrock 单作者的一项研究(尚未同行评审的 arXiv 预印本,https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.26584)对比了 6 个开源底座模型与各自的后训练版本,结论一句话:后训练改变的是模型读取表征的方式,不是表征本身。\n\n## 知识在,读法错了\n\n研究用三套德语语料:25 词以内截断的短聊天(300 条文本,出自 61 条人写原文)、本地报纸段落(103 段,出自 14 篇文章)、教科书摘要(238 条,119 对人机配对)。线性探针在网络三分之二深度处把两类文本分得很开:三个语料的中位 d′ 分别为 1.35、2.66、3.70。更扎心的是,只用词频的朴素分类器达到同样甚至更好的效果(短聊天 d′=1.73),连放在平均输入嵌入上的探针都不差——判别线索本来就摊在文本表面。但模型自己的答案大量接近随机,甚至反向:Qwen3-8B 后训练版判报纸段落的 d′ 低至 -1.01(原始提示)与 -1.30(聊天式提示),Gemma 3 12B 判短聊天为 -0.81 与 -1.09。信息在表征里,答案没有用它。\n\n## SFT 阶段,平均削掉 0.34\n\n六个模型家族的后训练版相对底座平均 Δd′=-0.34,六个家族全部下降;换成三种提示措辞再测,平均 -0.29,54 个对比中 39 个变差。给出明确的基础比率后,底座模型对数几率的移动小于先验变化,多数后训练版反而移动过头。语料边界也要小心:报纸段落上训出的探针拿去判短聊天,方向直接翻转(中位 d′=-0.46),同一批特征在一个语料里标记 AI,在另一个里标记人类。三个发布了分阶段 checkpoint 的模型上,下降主要发生在监督微调(SFT)阶段。\n\n## 变化藏在早期层和文本之后的位置\n\n模型拼接(把一个 checkpoint 的前 N 层接到另一个的上层)给出定位:Apertus 1.5 8B 共 32 层,前 8 层换成 SFT 版就足以废除底座的判别力(d′=-0.50),前 8 层换回 base 版则恢复到 +0.59(底座本身 +0.43);OLMo 2 7B 前 6 层 SFT 就造出全部亏损(-0.48)。变化在早期若干层渐进累积(Apertus 第 3-8 层,OLMo 2 第 1-14 层)。表征相似度同向:文本位置上 base 与后训练版的线性 CKA 为 0.97-0.999,答案形成位置只有 0.44-0.99。激活修补的跨层扫描进一步显示,从第 14 层起,是文本之后的位置(问题与答案提示)的内部状态在两个 checkpoint 之间传递这个变化;OLMo 2 里文本自身的状态在任何层都不起作用。探针跨版本迁移在文本位置的损失最多 0.15 d′,在答案位置的上三分之一深度却达 0.7-1.9——读出位置被重组,而不是被替换。\n\n## 反方向的案例:否定句\n\n作者身份语料无法共享,论文补了一组公开数据上可复现的否定陈述句判断:六个家族的探针都把真理解码到天花板(d′=4.54),但底座 OLMo 2 的答案只有 0.80,Apertus 底座干脆反向(-0.88),几乎永远答「假」(判断标准 c=2.6 与 3.0)。SFT 把两者都修复(3.36 与 4.54),探针保持天花板。修复同样从网络中段起位于句子之后的位置;中段以下,陈述本身的表征有贡献,OLMo 2 高达修复量的 82%,其余三个模型为 19-45%。后训练不总是「削」:底座读错的它修,底座已经读对的(Qwen3 与 OLMo 3 底座)基本不动。\n\n## 2000 条标签就能定方向\n\n训练研究直接操纵了信号:给 OLMo 2 的微调数据加 2000 条正确标注的新闻作者判断,答案在 held-out 新闻上 d′=4.28,报纸段落 2.18(随机标签对照 0.13),教科书摘要相对随机标签提升 +1.36。但同一批训练让短聊天判得比随机还差(-0.58)——新闻上学到的「AI 特征」到聊天语料上反噬,答案轨迹跟着底座模型上新闻训出的探针走。方向由训练信号决定,不是由「更对齐」决定。\n\n## 该泼的冷水\n\n语料不能共享(报纸未发表、聊天消息无授权、摘要有版权),否定句案例是作者提供的公开复现入口,代码与结果表在项目仓库(https:\u002F\u002Fgithub.com\u002Flaubrock-lab\u002Frepresented-but-misread)。拼接显示的是携带差异的权重在哪里,不是差异在哪里计算——杂交模型可能跑出两个亲本都到不了的值,作者用远离随机的对照判断排除了拼接本身的干扰,也承认没做「接近随机且未被改变」的理想对照。定位结论在作者身份任务上只基于两个模型;探针证明可解码不等于被使用,作者因此把主要结论压在拼接与修补上。\n\n## 所以呢\n\n这项研究给「表面对齐假说」添了因果证据:知识在预训练中获得,后训练安装的是读法——但读法并不浅,它动早期层权重、改变问题位置对文本的取用,并带来真实的判断质量后果。对工程的含义很直接:拿 chat 模型当 AI 文本检测器或裁判,等于用一个被系统性教歪过的读法在工作;修复不贵,2000 条标签就能调向,但方向跟着训练分布走,换语料可能反噬。机制层另有独立团队互证:Orgad 等人发现幻觉的真实性信息在内部表征里远多于口头输出所用(https:\u002F\u002Farxiv.org\u002Fabs\u002F2410.02707);Loos 与 Eberle 的 KV-cache 修补把微调行为集中定位在系统提示与聊天模板 token(https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.41480);Ackerman 与 Panickssery 发现 Llama3-8B 的自我识别能力是 post-training 之后才出现的(https:\u002F\u002Farxiv.org\u002Fabs\u002F2410.02064)。三支不同团队、三条路线,指向同一件事:后训练改的是读出,不是知识。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.26584","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"40269b40-7942-4650-9672-ed2e6524d37a","ai-technology",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"f72d264d-7fa0-458b-8d43-0ec5168d69db","instruct-model",{"id":19,"name":20,"slug":20,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":22,"name":23,"slug":23,"description":14,"color":14},"b9bd9039-fcdb-41a8-b85b-fc1587def2b9","open-source",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"e4e0f96a-82a8-4218-acc3-61f7c0fb4e7a","en","Post-training changes the readout, not what models know","Potsdam study: probes separate AI from human German text (d' up to 3.70), model answers invert after SFT, which carries the change in early layers.","Ask an open-weight language model whether a German text was written by a human or by an AI, and something odd happens: the information needed to answer is visible in the model's internals, yet the answer is often near chance - or inverted. A single-author study by Jochen Laubrock of the University of Potsdam (an arXiv preprint, not yet peer-reviewed, https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.26584) compared six open-weight base models with their post-trained versions. The upshot: post-training changes how models read out what they represent, not what they represent.\n\n## The knowledge is there; the readout misses it\n\nThree German corpora were used: short chat messages truncated to 25 words (300 texts from 61 human originals), local-newspaper segments (103 segments from 14 articles), and textbook summaries (238 texts, 119 matched human-AI pairs). Linear probes at two-thirds of network depth separated AI from human text well: median d' of 1.35, 2.66, and 3.70 across the three corpora. Plain word-frequency classifiers did as well or better (d' = 1.73 on the chats), and even a probe on the averaged input embedding was close - the cues sit on the surface of the text. Yet the models' own answers were often near chance or inverse: post-trained Qwen3-8B judged the newspaper segments at d' = -1.01 (raw prompt) and -1.30 (chat prompt), and Gemma 3 12B judged the short chats at -0.81 and -1.09. The information was in the representations; the answers did not use it.\n\n## SFT, on average, costs 0.34 of sensitivity\n\nAcross the six model families, post-training changed the sensitivity of the answers by Δd' = -0.34 on average, with a decrease in all six families; across three prompt wordings the mean was -0.29, with 39 of 54 comparisons worse. Given an explicit base rate, base models shifted their log-odds less than the stated change in prior odds, while most post-trained models overshot. Corpus boundaries matter: a probe trained on the newspaper segments judged the short chats with the wrong sign (median d' = -0.46) - the same surface features that mark AI text in one corpus mark human text in another. In the three models with stage-wise checkpoints, the drop occurred mainly during supervised fine-tuning (SFT).\n\n## Early layers, and the positions after the text\n\nModel stitching - lower layers of one checkpoint spliced under the upper layers of another - located the change. In Apertus 1.5 8B (32 layers), taking the first 8 layers from the SFT checkpoint was enough to abolish the base model's discrimination (d' = -0.50), while the first 8 base layers restored it under SFT upper layers (+0.59, against +0.43 for the unmodified base). In OLMo 2 7B, the first 6 SFT layers produced the full deficit (-0.48). The change builds up gradually across early layers (Apertus layers 3-8; OLMo 2 layers 1-14). Representation similarity agrees: linear CKA between base and post-trained checkpoints was 0.97-0.999 at text positions but 0.44-0.99 at the answer position. Layer-wise activation patching showed that from layer 14 onward, the internal states at the positions after the text - the question and the answer cue - carried the change between checkpoints; in OLMo 2 the text states had no effect at any layer. Probes transferred across versions at the text positions with a loss of at most 0.15 d', but at the answer position the deficit reached 0.7-1.9 in the upper third of the network: the readout position was reorganized, not replaced.\n\n## The opposite direction: negated statements\n\nThe authorship corpora cannot be shared, so the paper adds a case reproducible from public data: the truth of negated statements. Probes decoded the truth at ceiling (d' = 4.54) in all six families, but the base OLMo 2 answered at only 0.80, and the base Apertus ordered the statements inversely (-0.88), answering 'False' almost always (criterion c = 2.6 and 3.0). SFT repaired both (3.36 and 4.54) while the probes stayed at ceiling. The repair again sat at the positions after the statement from mid-depth on; below that, the representation of the statement contributed - up to 82% of the repair in OLMo 2, and 19-45% in the other three models. Post-training does not only subtract: it repairs what a base model misread, and leaves alone what base models already read correctly (Qwen3, OLMo 3).\n\n## 2,000 labels set the direction\n\nThe training study manipulated the signal directly: adding 2,000 correctly labeled authorship judgments of news texts to OLMo 2's fine-tuning data raised the answers to d' = 4.28 on held-out news and 2.18 on the newspaper segments (random-label control 0.13), and by +1.36 over random labels on the textbook summaries. The same training made the short chats worse than chance (-0.58): cues learned from news inverted on chat messages, and the answers followed a news-trained probe of the base model. The direction is set by the training signal, not by alignment as such.\n\n## Cold water\n\nThe authorship corpora cannot be shared (unpublished newspapers, chat messages without consent, copyrighted summaries); the negation case is the author's reproducible entry point on public data, with code and result tables in the project repository (https:\u002F\u002Fgithub.com\u002Flaubrock-lab\u002Frepresented-but-misread). Stitching shows where the weights carrying the difference reside, not where it is computed - hybrids can overshoot both parents, and the paper's control covers judgments far from chance, while the ideal control near chance was not run. The localization rests on two models for authorship; and a probe shows decodability, not use, which is why the central claim rests on stitching and patching.\n\n## So what\n\nThe study adds causal evidence to the superficial-alignment hypothesis: knowledge is acquired in pretraining, and post-training installs a readout - but the readout is not shallow. It involves early-layer weights, changes how the question positions take up the text, and has real consequences for judgment quality. The practical point is blunt: using a chat model as an AI-text detector or as a judge means using a systematically miscalibrated readout. Repair is cheap - 2,000 labels can turn it - but the direction follows the training distribution, and switching corpora can backfire. Independent teams converge on the mechanism: Orgad et al. found that internal representations encode far more truthfulness than verbalization uses (https:\u002F\u002Farxiv.org\u002Fabs\u002F2410.02707); Loos and Eberle's KV-cache patching concentrates fine-tuned behavior at system-prompt and chat-template tokens (https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.41480); and Ackerman and Panickssery found that Llama3-8B's self-recognition appears only after post-training (https:\u002F\u002Farxiv.org\u002Fabs\u002F2410.02064). Three teams, three routes, one conclusion: post-training changes the readout, not the knowledge.","post-training-changes-readout-not-representations","2026-11-20T17:10:34Z","2026-11-20T17:13:08.448896Z","2026-11-20T17:13:08.448907Z",true,"agent",42,[39,49,58],{"slug":40,"tag_slug":41,"title_zh":42,"title_en":43,"intro_zh":44,"intro_en":45,"id":46,"is_active":35,"created_at":47,"modified_at":48},"image-generation","text-to-image","图像生成：从文生图到逐框编辑的模型竞赛","Image Generation: from text-to-image to frame-level editing","文生图已是红海，战场正转向编辑与可控性：FLUX.2\u002F3 用开源和量化把生图搬进消费级显卡，Qwen-Image、Seedream 等国产模型紧咬不放，GPT Image 2 则把推理能力带进像素。本专题追踪图像生成与编辑模型的发布、架构演进、加速量化与商业化数据，看「好图」如何变成「可用的图」。","Text-to-image is a crowded race, and the frontier has moved to editing and control: FLUX.2\u002F3 push open weights and quantization onto consumer GPUs, Qwen-Image and Seedream close in from China, and GPT Image 2 brings reasoning into pixels. This topic tracks model releases, architecture shifts, acceleration work, and the business numbers behind \"pretty\" becoming \"usable\".","52de7c59-7864-4569-8ae7-3d656d26a1a1","2026-11-20T07:20:23.534506Z","2026-11-20T07:20:23.534527Z",{"slug":50,"tag_slug":50,"title_zh":51,"title_en":52,"intro_zh":53,"intro_en":54,"id":55,"is_active":35,"created_at":56,"modified_at":57},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":59,"tag_slug":59,"title_zh":60,"title_en":61,"intro_zh":62,"intro_en":63,"id":64,"is_active":35,"created_at":65,"modified_at":66},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":68},[69,74,79,84,89,94],{"id":70,"title":71,"news_slug":72,"published_at":73},"ef87ba13-75d5-41ff-a6d0-0a6e8b2cf30d","偏好标签三成翻错:PORT照拿67.6%胜率","port-token-reweighting-noisy-preference","2026-11-10T17:30:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"ba729d8d-d18e-4e77-824e-1db99ec469eb","近零词嵌入暗坑:五成门控检查点训练静默白跑","near-zero-embeddings-gated-training-stall","2026-11-04T15:20:36+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"eb55e1cf-15c1-487f-845c-e81f87f11cfa","红白机 CPU 编进 Qwen 权重:没训练,纯编译","nesformer-qwen-nes-cpu-weights","2026-10-31T17:30:00+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"a491d4db-2d5a-4723-b093-28d1512fd647","VLA-Δ:物理状态变化当监督信号,LIBERO 98.4","vla-delta-physical-state-transition","2026-10-20T19:07:27+00:00",{"id":90,"title":91,"news_slug":92,"published_at":93},"50687317-89b7-4857-ad66-86e2e251ff9d","阿里开源 OpenGCUA：8 个前沿模型电脑操控平均涨 19.6 分","opengcua-open-computer-use-stack","2026-10-18T13:10:17+00:00",{"id":95,"title":96,"news_slug":97,"published_at":98},"36e4f28f-9176-485c-92b2-35cd0aac04fc","Apple MLX 联合 Meta 把 816B 模型装进 Mac Studio 集群","apple-mlx-muse-spark-mac-studio-thunderbolt-rdma","2026-10-16T03:00:00+00:00"]