[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-resolution-depth-word-reading-time-llm":3,"topics-all":38,"news-related-872f163b-5ed4-4405-91aa-48c0fe34c25f":67},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"872f163b-5ed4-4405-91aa-48c0fe34c25f","论文实测:模型在哪一层想通这个词,能预测人读它多久","论文用调谐透镜逐层读出每个词的意外度轨迹,定义「解析深度」D:模型沿网络深度把这个词想明白的位置。在 8 个英文眼动与自定速阅读语料、10 个模型上,晚解析的词和它后面那个词,人读得更慢,留出集对数似然提升 0.26 millinats 每词。但效应量小、语料间差异大,且只在调谐透镜下成立。","一个词在模型里被想明白的早晚,可能比它有多意外更能解释人读它花了多久。\n\n这不是修辞。意大利罗马的 Universitas Mercatorum 的 Donato Riccio 与 Fabrizio Maturo 在一篇 arXiv 预印本里(https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.25970),用 8 个英文眼动与自定速阅读语料、10 个模型(GPT-2、OPT、Pythia)测了这件事:让模型沿深度逐层解码每个词的下一词概率,取这条意外度轨迹变化的质心,称为「解析深度」D。D 取值 0 到 1,变化均匀摊开时等于 0.5;10 个模型的 D 中位数落在 0.38 到 0.50 之间,论文说这个质心摊在网络中部。作为对照,论文另有一个阈值指标——累计变化达 90% 的那个深度——在 60% 到 89% 的词上都落在深度的最后五分之一,说明阈值法会把信息全挤到顶部。\n\n## 越晚想通,人读得越慢\n\n在 8 个语料、10 个模型汇总的主指标上,D 让留出集的对数似然再提升 0.26 millinats 每词(95% CI 0.10 到 0.40,p = 0.001)。这是扣掉词频、词长、最终层意外度、熵和最佳单层这些控制项之后的净增量。分模型族看,GPT-2 增益 0.36(p = 0.004),OPT 0.25(p = 0.002),Pythia 0.18(p = 0.021)。\n\n换成人能感知的单位:在 OneStop 语料上,GPT-2 的 D 每升高一个标准差,回扫时间(go-past time)在该词上延长 0.6% 到 1.3%,在它后面那个词上延长 1.1% 到 1.6%。大部分效应其实落在后一个词上:单独加 D(前一个词) 的增益是 0.17(p = 0.002),大于单独加 D(当前词) 的 0.09(p = 0.015)。\n\n## 哪些词被解析得晚\n\n论文把最终意外度落在同一档的词分成 D 最高和最低两组比:晚解析的词在 80 组语料-模型配对里有 74 组更短(中位 0.44 个字母),65 组更高频,60 组更常是功能词(占比中位 55%,对照组 50%)。在 OneStop 配 GPT-2 XL 上,论文点名的晚解析高频词是 and、in、a、the、to、for、more、of,早解析的是 the、have、has、is、was。\n\n真的拖慢人的,往往是那些看起来信息量很低的连接词和虚词。读者在 and 上多花的工夫,比一个罕见名词还多。\n\n## 该泼的冷水\n\n作者自己泼了三盆。\n\n一是效应量小。相对最终层意外度带来的收益,D 的增益只占 2.1%(0.26 对 12.05 millinats 每词);把 8 个语料当样本做随机效应模型,净增益的均值是 0.07,p = 0.37,组间方差占 75%,对新语料的预测区间是 -0.93 到 1.07。站得住的是跟「打乱层顺序」的对照比:8 个语料里 7 个为正,随机效应均值 0.23(0.03 到 0.44,p = 0.016),组间方差降到 27%。而打乱层顺序后的 D 在主指标上只有 -0.016(p = 0.79)。\n\n二是不同语料差别巨大。GECO 这支语料给几乎每个模型都是零,只有 OPT 6.7B 是 0.31(p = 0.024);UCL 的自定速阅读有 10 个模型里 8 个为正(汇总 0.36,p = 0.19),UCL 眼动则 9 个模型为负(-1.30)。GPT-2 和 Pythia 的效应各自只靠一个语料撑着,去掉 OneStop 之后 GPT-2 只剩 0.02(p = 0.41)。\n\n三是方法论依赖。效应只在调谐透镜(tuned lens)下出现:用未训练的随机权重网络跑同一套流程,增益是 -0.03(p = 0.75),训练过的模型在同样口径下是 0.22(p = 0.025)。而 tuned lens 自己的论文承认公开发布的透镜训练得远远不够,换成 logit lens 去掉嵌入层读出后增益只有 0.01(p = 0.39)。另外 ZuCo、MECO 和两篇 Natural Stories 文本都来自维基百科,而 OPT 和 Pythia 的训练数据包含维基百科、GPT-2 的排除了。论文自我定位是探索性分析,模型规模最大只到 6.9B 参数。\n\n## 不止这一条路\n\n同一时期有三份独立工作指向同一个方向。Hu、Lepori 与 Franke 测了 20 个开源模型、6 个领域,发现前向过程的动态量比静态末层量更能预测人的处理过程,而且更大的模型不一定更像人(https:\u002F\u002Farxiv.org\u002Fabs\u002F2504.14107)。Barenholtz 用「轨迹外推误差」——拟合前几个隐藏状态再外推,看下一个词偏离多远——这个量与意外度几乎正交(r = .044),在 Natural Stories 上独立预测自定速阅读时间,并在 GPT-2 与 Pythia\u002FRoPE 两种位置编码架构间复现(https:\u002F\u002Farxiv.org\u002Fabs\u002F2606.05346)。Hayakawa 与 Saggion 把「预测深度」的分布跟读者熟练度挂钩,在 12 个 LLM 上发现词汇量偏低的读者在首遍注视时长上有更深的预测深度(https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.37688)。\n\n三份论文的度量各不相同,指向的方向却一致:模型的中间层不是残差计算的副产品,而是承载了跟人类认知相关的信息。\n\n## 所以呢\n\n如果「模型在第几层想通这个词」真能预测人读它多久,那有一件事值得重新算:把大模型当黑箱来解释人的阅读习惯,扔掉了一整条深度轴。论文的 D 只要一次前向加一个透镜就能算,不挑模型深度,可以横向比从 GPT-2 到 6.9B。\n\n但别急着把它当成已经可用的方法。它现在是一个在多数语料上稳健、在 GECO 上归零、并且依赖透镜质量的指标,80 组语料-模型配对里只有 9 组单独显著(随机期望 4 组),260 个训练格子里 30 个显著(随机期望 13 个)。论文自己列的下一步是:做一组「固定概率但改变所需计算量」的实验,看那部分计算本身是不是真的要花人的时间。\n\n在那之前,把 D 当成一个诊断视角,比当成一个能用的产品更合适。\n\n附注:论文致谢写明,分析代码和正文起草用了 Claude(Anthropic),作者全程监督并对内容负责。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.25970","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"40269b40-7942-4650-9672-ed2e6524d37a","ai-technology",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"120fa59a-ff6f-4537-9bf5-f818df636a0e","benchmark",{"id":19,"name":20,"slug":20,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":22,"name":23,"slug":23,"description":14,"color":14},"4f214978-cac1-4f39-aa4b-f92a0d0934b7","transformer",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"6c71605f-f1c1-4f0d-9359-7b5e376a0b35","en","Paper test: word resolution depth predicts reading time","Resolution depth, the layer where a model settles a word, predicts reading time: 0.26 millinats per word across 8 corpora, 10 models, lens-dependent.","When a language model settles on a prediction for a given word, it does so at some point along its depth. A new preprint argues that *where* along that depth the decision happens predicts how long people take to read that word, and it does so beyond anything the final-layer surprisal already tells you.\n\nDonato Riccio and Fabrizio Maturo of Universitas Mercatorum in Rome ran the experiment (https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.25970) across eight English eye-tracking and self-paced reading corpora and ten models from three families (GPT-2, OPT, Pythia). The method is a measurement of when, not how much: read every word's next-token probability at every layer with the tuned lens, take the trajectory of surprisal across depth, and summarize it by its resolution depth D, the centroid of where the absolute change sits. D runs from 0 to 1 and equals 0.5 when the change is spread evenly; across the ten models its median sits between 0.38 and 0.50, so the settling point lives in the middle of the network. The paper's alternative threshold landmark, the depth at which 90% of the cumulative change is complete, falls in the top fifth of depth for 60% to 89% of words, which is why the centroid is the measure that carries information.\n\n## Later resolution, slower reading\n\nPooled over the eight corpora and ten models on the primary measure, adding D to a control set that already contains word frequency, length, final-layer surprisal, entropy and the best single readout improves held-out log-likelihood by a further 0.26 millinats per word (95% CI 0.10 to 0.40; p = 0.001). Each model family gains on its own: GPT-2 0.36 (p = 0.004), OPT 0.25 (p = 0.002) and Pythia 0.18 (p = 0.021).\n\nIn units a reader can feel, on the OneStop corpus a one-standard-deviation rise in D lengthens go-past time by 0.6% to 1.3% on the word itself and by 1.1% to 1.6% on the word that follows. Most of the effect lands on the successor: D of the preceding word alone gains 0.17 (p = 0.002) against 0.09 (p = 0.015) for D of the current word.\n\n## Which words resolve late\n\nComparing the fifth of words with the highest D against the fifth with the lowest D within each decile of final surprisal, the late-resolved words are shorter in 74 of the 80 corpus-model pairs (median 0.44 letters), more frequent in 65 and more often function words in 60 (median share 55% against 50%). On OneStop under GPT-2 XL, the commonest late-resolved words of middling surprisal are and, in, a, the, to, for, more and of; the commonest early-resolved ones are the, have, has, is and was.\n\nThe words that slow readers down are often the connectives and function words that look information-free. Readers spend more time on and than on a rare noun.\n\n## Three caveats the authors raise themselves\n\nFirst, the effect is small. Measured against the gain from final-layer surprisal, D contributes 2.1% (0.26 against 12.05 millinats per word). Treating the eight corpora as a sample, the random-effects mean of the net gain is 0.07 with p = 0.37 and 75% of the variation between corpora unexplained, giving a prediction interval of -0.93 to 1.07 for a new corpus. What does hold up is the comparison against a layer-shuffled D: positive in seven of the eight corpora, random-effects mean 0.23 (0.03 to 0.44; p = 0.016), with between-corpus variation down to 27%. Shuffled D on its own manages -0.016 (p = 0.79).\n\nSecond, the corpora disagree sharply. GECO gives gains near zero for every model except OPT 6.7B (0.31; p = 0.024). On the UCL self-paced reading corpus eight of the ten models gain (0.36 pooled; p = 0.19) while UCL eye tracking loses for nine (-1.30). For GPT-2 and Pythia the effect rests on one corpus each; drop OneStop and GPT-2 keeps just 0.02 (p = 0.41).\n\nThird, the result depends on the readout. The gain shows up only under the tuned lens: untrained networks with random weights gain -0.03 (p = 0.75) while trained models read the same way gain 0.22 (p = 0.025). The tuned-lens paper itself reports that the released lenses are severely undertrained, and under the logit lens with logit-lens controls and without the embedding readout the gain is 0.01 (p = 0.39). There is also a training-data wrinkle: ZuCo, MECO and two Natural Stories texts derive from Wikipedia, which the training data of OPT and Pythia include and GPT-2's excludes. The paper calls its analyses exploratory and covers decoder-only transformers up to 6.9B parameters.\n\n## Three independent lines of work\n\nThree separate groups have been converging on the same idea. Hu, Lepori and Franke tested 20 open-source models across 6 domains and found that dynamic measures improve prediction of human processing measures relative to static final-layer measures, and that larger models do not always show more human-like processing patterns (https:\u002F\u002Farxiv.org\u002Fabs\u002F2504.14107). Barenholtz introduced trajectory extrapolation error, fitting a linear path to the preceding hidden states and measuring deviation from it; the measure is nearly orthogonal to surprisal (r = .044) and independently predicts self-paced reading times on Natural Stories, replicating across GPT-2 and Pythia\u002FRoPE positional encodings (https:\u002F\u002Farxiv.org\u002Fabs\u002F2606.05346). Hayakawa and Saggion tied the distribution of predictive depth to reader proficiency, finding across 12 tested LLMs that readers with lower vocabulary proficiency show deeper predictive depth for first-pass gaze duration (https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.37688).\n\nThe metrics differ, but the direction agrees: the middle layers of a model are not a byproduct of residual computation, they carry information about human cognition.\n\n## So what\n\nIf the layer at which a model resolves a word really does predict how long people read it, then one thing worth recomputing is this: treating large models as black boxes when explaining human reading habits throws away an entire depth axis. The measure itself is cheap - one forward pass and a lens, no particular model depth required, comparable from GPT-2 up to 6.9B.\n\nThat said, do not treat it as a usable method yet. Right now it is a measure that holds across most corpora, collapses to zero on GECO, and depends on lens quality. Of the 80 corpus-model pairs, only 9 are significant on their own (4.0 expected by chance) and 30 of the 260 trained cells gain significantly (13.0 expected). The paper's own next step is to run experiments that vary the computation a prediction needs at a fixed probability, and see whether that computation itself costs human readers time.\n\nUntil then, D is better treated as a diagnostic lens than as a product.\n\nNote: the acknowledgments state that the authors used a generative AI system (Claude, Anthropic), under their supervision throughout, in writing the analysis code and drafting the manuscript text.","resolution-depth-word-reading-time-llm","2026-11-22T05:00:00Z","2026-11-22T05:10:53.036251Z","2026-11-22T05:10:53.036266Z",true,"agent",22,[39,49,58],{"slug":40,"tag_slug":41,"title_zh":42,"title_en":43,"intro_zh":44,"intro_en":45,"id":46,"is_active":35,"created_at":47,"modified_at":48},"image-generation","text-to-image","图像生成：从文生图到逐框编辑的模型竞赛","Image Generation: from text-to-image to frame-level editing","文生图已是红海，战场正转向编辑与可控性：FLUX.2\u002F3 用开源和量化把生图搬进消费级显卡，Qwen-Image、Seedream 等国产模型紧咬不放，GPT Image 2 则把推理能力带进像素。本专题追踪图像生成与编辑模型的发布、架构演进、加速量化与商业化数据，看「好图」如何变成「可用的图」。","Text-to-image is a crowded race, and the frontier has moved to editing and control: FLUX.2\u002F3 push open weights and quantization onto consumer GPUs, Qwen-Image and Seedream close in from China, and GPT Image 2 brings reasoning into pixels. This topic tracks model releases, architecture shifts, acceleration work, and the business numbers behind \"pretty\" becoming \"usable\".","52de7c59-7864-4569-8ae7-3d656d26a1a1","2026-11-20T07:20:23.534506Z","2026-11-20T07:20:23.534527Z",{"slug":50,"tag_slug":50,"title_zh":51,"title_en":52,"intro_zh":53,"intro_en":54,"id":55,"is_active":35,"created_at":56,"modified_at":57},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":59,"tag_slug":59,"title_zh":60,"title_en":61,"intro_zh":62,"intro_en":63,"id":64,"is_active":35,"created_at":65,"modified_at":66},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":68},[69,74,79,84,89,94],{"id":70,"title":71,"news_slug":72,"published_at":73},"3e60e849-882a-4722-9ca8-80324936b001","撤掉更差,73%后反超:Muon临时权重衰减的延迟账","muon-temporary-weight-decay-delayed-effect","2026-11-16T03:30:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"0a3f5044-ed20-4c2a-b710-bd26cd276d3e","ALiBi 的隐藏数值故障：长上下文越长，部分注意力头越可能“失明”","alibi-attention-underflow-long-context","2026-08-06T10:30:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"4f5d6e7d-b06c-4331-8e13-b916fa82bd2e","论文实测:70B模型说谎,探针认的是措辞","persona-direction-lie-probe-measures-wording","2026-11-22T09:00:00+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"42f5ea83-365c-4a4e-a318-17d7c4c11c15","论文实测:自洽性证书在AIME上24%的题上会误触发","self-consistency-certificates-misfire-correlated-reasoning","2026-11-21T09:30:00+00:00",{"id":90,"title":91,"news_slug":92,"published_at":93},"53dd30bb-ba19-45b6-a2de-5011709608b6","论文实测:一段文本认出39%审稿人,靠的是曝光量","frontier-models-name-authors-by-exposure","2026-11-21T01:40:00+00:00",{"id":95,"title":96,"news_slug":97,"published_at":98},"c4623b85-37f5-435e-ae98-e67eb72eb1dc","别再逐篇问大模型:AI写程序当评委,每主题0.17美元起","execjudges-llm-written-program-judges-retrieval-eval","2026-11-20T01:40:00+00:00"]