[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-v-lens-unified-model-hidden-state-visualization":3,"topics-all":38,"news-related-7726a377-f3e3-4fd1-93eb-6c4b075b685f":67},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"7726a377-f3e3-4fd1-93eb-6c4b075b685f","论文实测:统一模型自画隐藏状态,代词可视化最高94%","一篇 arXiv 论文(2611.24793)让统一多模态模型把自己的隐藏状态画成图:借模型自带的图像生成器,不用额外训解码器。六个模型上,代词指代的正确可视化率经 lens tuning 从 2%-11% 升到 86%-94%,纯视觉监督还顺带改善了文字答案,但每次读取要花 9 到 110 秒。","问题问「最大的陆地动物住在哪个大洲」,模型直接答 Africa。但它从头到尾没说过 elephant 这个词,大象这个概念只在某个中间状态里待过一阵。现在能把那个状态直接画出来,还不用训练额外的解码器。\n\n## 一、怎么做到的:借模型自己的画笔\n\n论文的做法叫 V-lens(arXiv:2611.24793,见 https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.24793,标题 The Elephant in the Question,作者 Tobias Braun、Marcus Rohrbach、Lorenzo Cavallaro,分别来自 Technische Universität Darmstadt、University College London 和 Mohamed bin Zayed University of Artificial Intelligence)。链路只有三步。\n\n先定插入层 R,也就是哪一层最适合被图像生成器读懂。靠标定:论文用 12 个视觉分得开的常见名词,要求每个模型 12\u002F12 全恢复。六个模型的插入层落在 BAGEL 8\u002F28、Show-o2 27\u002F28、SenseNova-U1 24\u002F42、HunyuanImage-3.0 9\u002F32、Liquid 6\u002F28、Janus-Pro 8\u002F30(层号\u002F总层数)。\n\n再把要看的那个 token 的隐藏状态搬过去:从抽取层取出状态,按目标层的均值做中心化,按两层的离散程度缩放,让尺度落在生成器认得的范围内。\n\n最后塞进一句普通绘图请求里的空槽:状态放在「一张 thing 的照片」的 thing 位置,后面接上模型自己的图像生成器,图就出来了。\n\n这一步很关键。论文在两跳题的测试集上对比变体(三个模型平均):只给词不给状态,正确率 0.03;只做最终归一化是 0.11;原样把状态拷过去 0.25;带 Jacobian 映射 0.27;完整的 V-lens 是 0.33。\n\n## 二、画出来的东西有多准\n\n第一类测试故意不出现物体名,两个描述还共用同一个末词 trunk(有长鼻子的动物、有粗树干的高植株),读它的隐藏状态,一个该画出大象,一个该画出树。30 对描述里,BAGEL 只对 2\u002F30,随机水平 0.03,p 值 0.25;Show-o2 8\u002F30(p=0.004);SenseNova-U1 4\u002F30(p=0.064);HunyuanImage-3.0 6\u002F30(p=0.016)。\n\n第二类是同一个词在句子里有两种意思,比如 seal 可以是海豹也可以是印章。20 个词里 BAGEL 12.5\u002F20,其余三个模型都是 15\u002F20,p 值全是 0.0001。\n\n打分靠三个多模态评委 Qwen3-VL、Gemma-3、LLaVA-OneVision-2。一张图要两个评委认出主体,一个状态要三个种子里两个成功。评委也做过校准:标注集上判断对 146\u002F156 张,带背景干扰的 67\u002F72 张也对;600 张抽样里,三名人类评分者与评委一致率都至少 88%。\n\n深度上的结果更扎人。Show-o2 在第 24 层还能从 15 个问题里的 11 个读出概念,到第 27 层(马上要吐答案时)掉到 0\u002F15,预设差值 0.73,95% 区间 [0.53, 0.93]。概念在答案落地那一刻被抹掉。\n\n更狠的测试:让模型一边想着某个东西一边照抄无关的句子。Show-o2 抄本里 the 在第 25 层画出自行车,三个种子全中;换成想玫瑰或不给指令,三个种子全不中。HunyuanImage-3.0 的句号 token 在第 25 层画出面包卷。\n\n## 三、lens tuning:画出来还能拿来训\n\n既然画得出来,就能拿画当监督信号:论文给文本层挂上秩 8 的 LoRA,用模型自己画的图监督某个隐藏状态,这叫 lens tuning。\n\n代词指代这个任务上,未训练时 BAGEL 是 0\u002F20 对,SenseNova-U1 4\u002F20,HunyuanImage-3.0 0\u002F20,Show-o2 本身就有 16\u002F20。训完之后在 120 对测试题、三个种子上,BAGEL 从 0.02 到 0.86,SenseNova-U1 从 0.11 到 0.94,HunyuanImage-3.0 从 0.06 到 0.86,论文口径是从 2%-11% 升到 86%-94%。\n\n更意外的是纯视觉监督还改善了文字答案。80 个训练故事上,Show-o2 的文字正确率从 36.3 涨到 49.6,配对的图片正确率从 52.5 涨到 73.3;HunyuanImage-3.0 是文字 40.0 到 50.8,图片 37.5 到 64.6。\n\n再把训练后模型的主题状态搬回原模型,正确率从 41\u002F56 升到 47-49\u002F56,搬走了 21%-39% 的余量增益。对照组也在:普通绘图监督会把 BAGEL 的 GenEval 拉低 23.3 个点,说明 lens tuning 的收益不是多训出来的副作用。通用能力上,GenEval 和 MMLU-Pro 都没显著损失,六个区间都盖住 0。\n\n## 四、该泼的冷水\n\n这套东西还不能当实时工具用:每读一个状态都要生成一整张图,论文自己测是 9 到 110 秒。\n\n题集也小:两跳题 16 道,描述对 30 对,两义词和代词对各 20 个,两状态对 16 对,状态场景 40 个,互换场景 28 个。论文在局限一节里写了,未命名物体的恢复本来就有限,在 BAGEL 和 SenseNova-U1 上都不显著高于随机。\n\n方法上也有自证问题:lens tuning 用的正是评价自己的那套透镜。不带图片的词状态监督,在 SenseNova-U1 的未见短语、HunyuanImage-3.0 的未见名词和短语上,反而超过 lens tuning。另一个细节:混元喂错图的对照组,文字正确率也从 40.0 涨到 47.5——视觉监督改善文字,不完全依赖图配对正确。\n\n实际收益上,文字答案提升大约是 80 个故事里多对 10 个,而且只在被监督过的故事上测。状态搬运搬的是整个 hidden state,没有配对的对照 donor,没法把功劳单独算给图里的内容。通用能力检查只跑了一个种子。\n\n## 五、往前看一步\n\n机制上这不是从零开始。Diffusion Lens(ACL 2024,见 https:\u002F\u002Faclanthology.org\u002F2024.acl-long.524\u002F)就在文生图的文本编码器上做过类似的事:从中间表示生成图像来解读它,还发现不常见概念要花更多计算。区别是它要外部文本编码器和单独的扩散模型,V-lens 用模型自带的生成器。\n\n也有前车之鉴。Faithful-Patchscopes(见 https:\u002F\u002Farxiv.org\u002Fabs\u002F2602.00300)发现,用模型自己解码隐藏表示会被语言先验带偏:上下文编码的是紫色西兰花,解码出的解释还是绿色,平均忠实度掉了 18.84%。V-lens 换了解码通道,天花板却还在:生成器只会画它会画的东西,读不出不等于脑子里没有。\n\n所以呢?统一多模态模型的稀缺资源,可能不是它会画图,而是它顺手给了你一把读内部状态的放大镜。只是这把放大镜还得现拍照片,慢到只能做离线审计。等它能实时读,「模型在想什么」大概才第一次可操作。\n\n代码在 https:\u002F\u002Fgithub.com\u002Fmultimodal-ai-lab\u002FV-lens 放出,仓库含各模型适配器、评委面板和训练脚本,按 README 不含图片、结果和权重。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.24793","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"5e628969-6d2a-437f-998a-104e4b16cfb1","ai-progress",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"40269b40-7942-4650-9672-ed2e6524d37a","ai-technology",{"id":19,"name":20,"slug":20,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":22,"name":23,"slug":23,"description":14,"color":14},"499f4b56-819d-49a3-9609-33e775143b86","multimodal",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"bc1e89f1-87aa-41ab-9e54-bd03ec26962c","en","V-lens: Unified Models Draw Their Own Hidden States","Unified models draw their own hidden states, no extra decoder. Across six models, correct pronoun visualizations rise from 2-11% to 86-94%.","Ask a model which continent the biggest land animal lives on and it answers Africa. But it never says the word elephant. The elephant concept exists only in some intermediate state along the way. Now there is a way to draw that state, without training any extra decoder.\n\n## How the drawing works\n\nThe paper is called V-lens (arXiv:2611.24793, https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.24793, titled The Elephant in the Question, by Tobias Braun, Marcus Rohrbach and Lorenzo Cavallaro of Technische Universität Darmstadt, University College London and Mohamed bin Zayed University of Artificial Intelligence). Three steps.\n\nFirst it fixes an insertion layer R, the layer whose states the image generator can still read. That layer is calibrated, not guessed: the authors pick 12 visually distinct common nouns and require every model to recover all 12 of 12. The chosen layers are BAGEL 8\u002F28, Show-o2 27\u002F28, SenseNova-U1 24\u002F42, HunyuanImage-3.0 9\u002F32, Liquid 6\u002F28 and Janus-Pro 8\u002F30, where the first number is the layer and the second the total layer count.\n\nSecond it carries the hidden state of a chosen token over. The state is taken from an extraction layer, centred on the target layer mean and rescaled by the spread of the two layers, so its scale lands where the generator recognises it.\n\nThird it drops the state into a slot of an ordinary drawing request. The state goes into the thing slot of a request like a photo of a thing, and the model's own image generator does the rest.\n\nThat third step matters. On the two-hop test set, averaged over three models, giving only the word and no state scores 0.03; adding just the final normalisation gives 0.11; copying the state unchanged gives 0.25; a Jacobian map gives 0.27; the full V-lens gives 0.33.\n\n## How accurate the pictures are\n\nThe first test deliberately leaves the object unnamed, and the two descriptions even share the same last word: an animal with a long trunk and a tall plant with a thick trunk both end in trunk. Reading that hidden state should give an elephant in one case and a tree in the other. Over 30 description pairs, BAGEL gets 2\u002F30 right against a chance level of 0.03 with p=0.25; Show-o2 gets 8\u002F30 with p=0.004; SenseNova-U1 4\u002F30 with p=0.064; HunyuanImage-3.0 6\u002F30 with p=0.016.\n\nThe second test is easier: one word with two meanings, such as seal as the animal or as the stamp, and the picture has to follow the context. Over 20 words BAGEL reaches 12.5\u002F20 and the other three models all reach 15\u002F20, with p=0.0001 throughout.\n\nScoring runs through three multimodal judges, Qwen3-VL, Gemma-3 and LLaVA-OneVision-2. A picture counts when at least two of them identify the subject, and a state counts when at least two of three drawing seeds succeed. The judges were calibrated too: on a labelled set the automatic judgement is right on 146\u002F156 pictures, including 67\u002F72 with background distractors, and three human raters each agree with the automatic judgement on at least 88% of 600 sampled pictures.\n\nThe depth result is the sharpest. In Show-o2 the concept is still recoverable on 11 of 15 questions at layer 24, then drops to 0\u002F15 at layer 27, right before the answer is emitted. The prespecified difference is 0.73 with a 95% interval of [0.53, 0.93]. The concept is wiped out at the moment the answer lands.\n\nAn even harsher test asks the model to think about one thing while copying an unrelated sentence verbatim. In Show-o2 the word the inside the copied sentence draws a bicycle at layer 25 in all three seeds when the model is told to think about a bicycle, and in no seed when it is told to think about a rose or given no such instruction. In HunyuanImage-3.0 the final period token of the copy reveals a bread roll at layer 25.\n\n## Lens tuning: the pictures can train the model\n\nIf a state can be drawn, it can also serve as a supervision signal. The authors attach rank-8 LoRA adapters to the text layers and supervise a hidden state with pictures the frozen model draws itself. They call this lens tuning.\n\nOn pronoun reference, the untrained models are mostly blind: BAGEL 0\u002F20 pairs, SenseNova-U1 4\u002F20, HunyuanImage-3.0 0\u002F20, while Show-o2 already sits at 16\u002F20. After tuning, over 120 test pairs and three seeds, BAGEL goes from 0.02 to 0.86, SenseNova-U1 from 0.11 to 0.94 and HunyuanImage-3.0 from 0.06 to 0.86, which is the rise from 2-11% to 86-94% the paper reports.\n\nMore surprising is that purely visual supervision also improves text answers. Over 80 training stories, Show-o2 correct text answers rise from 36.3 to 49.6 with paired correct pictures rising from 52.5 to 73.3; HunyuanImage-3.0 goes from 40.0 to 50.8 on text and from 37.5 to 64.6 on pictures.\n\nSwapping the trained subject state back into the original model lifts correct answers from 41\u002F56 to 47-49\u002F56, carrying 21% to 39% of the margin gain. The control arm is there too: ordinary drawing supervision lowers GenEval by 23.3 points in BAGEL, so the gain from lens tuning is not just extra training. On general capability, GenEval and MMLU-Pro show no significant loss, and all six intervals include zero.\n\n## The cold water\n\nThis is not a real-time tool yet: every reading requires generating a full image, and the paper measures 9 to 110 seconds per reading.\n\nThe item sets are small, mostly hand written: 16 two-hop questions, 30 description pairs, 20 two-meaning words, 20 pronoun pairs, 16 two-state pairs, 40 state scenarios and 28 interchange scenarios. The limitations section says plainly that recovery of unnamed objects is modest and is not significantly above chance in BAGEL or SenseNova-U1.\n\nThere is also a self-referential problem: lens tuning trains hidden states through the same lens used to evaluate them. And word state supervision without pictures beats lens tuning on unseen phrases in SenseNova-U1 and on unseen nouns and phrases in HunyuanImage-3.0. One more detail: in the wrong-picture control arm for HunyuanImage-3.0, correct text answers also rise, from 40.0 to 47.5, so the text improvement does not depend entirely on the picture being right.\n\nIn terms of actual payoff, the text answer gain is about ten extra stories out of 80, measured only on the supervised stories. The interchange test moves whole hidden states and has no control donor trained on different picture targets, so it does not isolate what the pictures contributed. The general capability checks use a single training seed.\n\n## What comes next\n\nThe mechanism is not new from scratch. Diffusion Lens, published at ACL 2024 (see https:\u002F\u002Faclanthology.org\u002F2024.acl-long.524\u002F), already read text-to-image text encoders by generating images from intermediate representations, and found that uncommon concepts require further computation compared to common ones. The difference is that it needs an external text encoder and a separate diffusion model, while V-lens uses the generator that the unified model already carries.\n\nThere is also prior art warning about the same failure mode. Faithful-Patchscopes (see https:\u002F\u002Farxiv.org\u002Fabs\u002F2602.00300) found that when a language model decodes its own hidden representations, inherent linguistic patterns can override the contextual information encoded in them: a hidden state that encodes the attribute purple for broccoli still produces green in the explanation, and faithfulness drops by 18.84% on average. V-lens swaps the decoding channel, but the ceiling stays: the generator can only draw what it knows how to draw, and a missing picture does not establish that the hidden state lacks the concept.\n\nSo what is the real prize here? Not that the model can draw. It is that a unified model hands you a magnifying glass for its own internals. The catch is that the glass still has to take a photograph, at 9 to 110 seconds per shot, which is fine for offline auditing and useless for live debugging. Once that becomes cheap, the question of what the model is actually thinking stops being a metaphor.\n\nThe code is at https:\u002F\u002Fgithub.com\u002Fmultimodal-ai-lab\u002FV-lens. The repository ships adapters for the models, the judge panel and the lens tuning training scripts, but as the README states it carries no pictures, results or weights.","v-lens-unified-model-hidden-state-visualization","2026-11-20T05:30:00Z","2026-11-20T05:25:34.031544Z","2026-11-20T05:25:34.031551Z",true,"agent",62,[39,49,58],{"slug":40,"tag_slug":41,"title_zh":42,"title_en":43,"intro_zh":44,"intro_en":45,"id":46,"is_active":35,"created_at":47,"modified_at":48},"image-generation","text-to-image","图像生成：从文生图到逐框编辑的模型竞赛","Image Generation: from text-to-image to frame-level editing","文生图已是红海，战场正转向编辑与可控性：FLUX.2\u002F3 用开源和量化把生图搬进消费级显卡，Qwen-Image、Seedream 等国产模型紧咬不放，GPT Image 2 则把推理能力带进像素。本专题追踪图像生成与编辑模型的发布、架构演进、加速量化与商业化数据，看「好图」如何变成「可用的图」。","Text-to-image is a crowded race, and the frontier has moved to editing and control: FLUX.2\u002F3 push open weights and quantization onto consumer GPUs, Qwen-Image and Seedream close in from China, and GPT Image 2 brings reasoning into pixels. This topic tracks model releases, architecture shifts, acceleration work, and the business numbers behind \"pretty\" becoming \"usable\".","52de7c59-7864-4569-8ae7-3d656d26a1a1","2026-11-20T07:20:23.534506Z","2026-11-20T07:20:23.534527Z",{"slug":50,"tag_slug":50,"title_zh":51,"title_en":52,"intro_zh":53,"intro_en":54,"id":55,"is_active":35,"created_at":56,"modified_at":57},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":59,"tag_slug":59,"title_zh":60,"title_en":61,"intro_zh":62,"intro_en":63,"id":64,"is_active":35,"created_at":65,"modified_at":66},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":68},[69,74,79,84,89,94],{"id":70,"title":71,"news_slug":72,"published_at":73},"0b9b2439-b5f8-413e-8e62-58621123cc2f","Continuous Audio Thinking：把「思考」搬进音频 LLM，零解码成本补齐声学信息损失","coat-continuous-audio-thinking-latent","2026-06-18T06:00:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"efca492c-b9b1-4d72-bf3b-32f08fc0f515","世界模型崛起：AI 从数字世界走向物理世界的关键一步","mit-world-models-physical-ai-lecun-fei-fei","2026-04-27T13:05:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"8f0348fd-b4d2-4567-8dae-f2d6f9cf7760","论文实测:算术电路焊进LLM权重,6位数冲上100%","zero-gradient-arithmetic-circuits-llm-weights","2026-11-21T17:30:00+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"42f5ea83-365c-4a4e-a318-17d7c4c11c15","论文实测:自洽性证书在AIME上24%的题上会误触发","self-consistency-certificates-misfire-correlated-reasoning","2026-11-21T09:30:00+00:00",{"id":90,"title":91,"news_slug":92,"published_at":93},"e9ce0c9c-c7f3-48ac-b4e8-bf58587dd451","删掉有害数据就防住了?论文说模型越强补回来越省","filtered-training-data-capability-returns-cheaper-with-scale","2026-11-19T05:30:00+00:00",{"id":95,"title":96,"news_slug":97,"published_at":98},"648ee35f-321e-4cc7-8e88-9a9d3ebae033","小模型每晚自学:一年后从 52 分掉到 5 分","on-device-llm-self-training-judge-failure","2026-11-19T03:30:00+00:00"]