[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-glance-generator-reranker-shared-prefill":3,"topics-all":38,"news-related-9cfac3fd-f359-467e-94ba-f8ed3172d713":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"9cfac3fd-f359-467e-94ba-f8ed3172d713","生成器自己当重排器:RAG 少驻留一个模型","论文Glance让RAG系统里的生成器兼任重排器不用再多驻留一个模型,一次共享前向里给每个候选段落单独打一个相关分,微调39分钟后重排基准得分0.741对照交叉编码器0.721,下游问答最终成绩高出0.045,作者自认这是概念验证单查询延迟仍高于专用交叉编码器。","RAG 系统里有一笔默认架构账:检索之后挂一个独立重排模型给段落打分,再挂一个生成模型读段落写答案,候选段落要被编码两遍,显存里也要同时驻留两份模型权重。Airun Intelligence Labs 10 月 29 日提交到 arXiv 的论文 Glance 提出一条更省的路径:反正生成器接下来就要读这些段落,不如让它自己给所有候选打分([论文](https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.40454))。\n\n## 机制:一次共享 prefill,每段一条短探针\n\nGlance 在一个冻结的 9B 骨干(MiMo-V2.6-Distill-Qwen-9B,混合门控 DeltaNet 线性注意力与全注意力层,比例 3:1)上挂了 rank-16 LoRA 适配器加一个 256 维指针头,这就是全部可训练参数。具体做法是把 k 个候选段落拼进同一个共享上下文,再为每段发一条约 30 token 的相关性探针,探针从共享缓存分叉,一条探针产出一个分数:段落只 prefill 一次,不再逐对编码。\n\n因为每条探针都能看到全部候选,作者推测它能拿一段和邻居段落做对比,这是孤立打分做不到的。打分结束关掉适配器,同一份骨干立刻变回普通聊天模型,模式切换实测 4.2–9.4 毫秒、输出概率逐位一致,显存驻留 16.85 GiB;每多挂一个决策技能只增加约 0.1 GiB。\n\n论文还测了一个更省的变体:适配器只在探针 token 上激活,段落编码仍走基座权重,答案生成就能直接续写重排时留下的缓存,段落真正只读一次。100 道 HotpotQA 题上,共享状态前向 64 毫秒、10 条探针 85 毫秒,续写缓存作答比重新 prefill 前三段更快(138 对 181 毫秒),F1 0.750 对 0.754 基本持平。作者标注这是 100 题、单随机种子的结果。\n\n## 数字账:39 分钟训练,自报分数略高于交叉编码器\n\n在 MS MARCO 上微调 39 分钟(4,000 条查询、55,598 个标注探针,单块 B200,峰值显存 21 GiB)后,论文报告 SciFact nDCG@10 0.741、NFCorpus 0.345,对照专用交叉编码器 bge-reranker-v2-m3 的 0.721 与 0.326;再加 14 分钟长上下文训练,两榜到 0.749 与 0.346。\n\n作者自己给这组数字踩了刹车:BEIR 上的差异没做显著性检验,样本只有约 300 条查询时,小于 0.02 的差距只能算指示性。换句话说,0.741 对 0.721 的领先按论文自己的标准就该谨慎读。\n\n更有分量的是下游。HotpotQA 300 题上,同一个 9B 生成器从 Glance 选出的前三段作答,F1 为 0.729,从交叉编码器选出的前三段作答是 0.684,差 +0.045(配对 bootstrap 95% 置信区间 [0.014, 0.076]),金段落召回 R@2 也更高(0.897 对 0.863)。\n\n## 反直觉点:召回更低,答案反而更好\n\n零样本那组实验有个值得记下的现象:Glance 的金段落召回明显低于交叉编码器(R@2 0.762 对 0.863),但生成器从它选的段落作答,F1 反而更高(0.718 对 0.683)。作者的解释是生成器挑的是自己用得上的上下文,但论文紧接着自我刹车:训练标签是段落相关性,不是答案质量,这不能说明模型学会了答案效用。\n\n## 诚实的负结果清单\n\n- 速度:专用交叉编码器每查询 31–60 毫秒,Glance 训练后在 B200 上约 420 毫秒,论文和模型卡都明说交叉编码器每查询仍然更快;\n- 内存:当前实现里每个探针分支要复制一份共享缓存,内存按候选数乘上下文长度增长,100 个候选时峰值 56 GiB;200 个候选(4.34 万 token)放不进单次调用,走分块路径要 15.8 秒;\n- 零样本适配器超过 20 个候选就退化,200 个候选时塌到 0.505;训练过的适配器反而随候选数上升继续涨(200 个候选时 0.717,对照交叉编码器 0.659);\n- 跨查询批处理目前反而降吞吐(每秒 2.87 降到 1.57);\n- 适配器特化伴随遗忘:通用决策基准 JevBench 从 184\u002F231 掉到 175\u002F231,丢的 9 题里 7 题在困难档;\n- 段落排列顺序会影响打分:不同顺序下 nDCG@10 在 0.678 到 0.693 之间波动,排序第一的段落有 91% 与原顺序一致——这是探针看得见邻居的代价。\n\n限制一节作者把话说得很透:这是概念验证,评估只覆盖两个检索数据集、一个问答集、一个骨干族。工程侧他们搭了一个 vLLM 原型,同一个引擎里聊天请求不带适配器、决策行带适配器、共享前缀缓存:20 个打包请求的输出与自研运行时秩相关 0.995,吞吐 1.54 对 0.86 请求每秒——生产化路径存在,但概率还有 0.006 的平均偏差。\n\n## 所以呢\n\n这份工作真正值钱的不是 0.741 这个分数,而是把「多驻留一个重排模型」这笔真实的显存与处理账摆上桌面,并给出一条 LoRA 级替代路径:一个骨干、多个 0.1 GiB 的决策技能、按请求切换。速度与内存工程都还没做,现在就在生产里砍掉重排器显然过早;适配器与代码已按 Apache-2.0 在 [Hugging Face 开源](https:\u002F\u002Fhuggingface.co\u002Fairun-ai\u002FGlanceRAG-MiMo-9B-Reranker)。对做 RAG 服务的人,更实际的启发或许是那个反直觉发现:重排器优化的段落相关性,和生成器真正需要的东西,可能本来就不是同一个目标函数。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.40454","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":19,"name":20,"slug":20,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":22,"name":23,"slug":23,"description":14,"color":14},"b9bd9039-fcdb-41a8-b85b-fc1587def2b9","open-source",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"b67491a2-fdde-441a-b71c-aba7ad828613","en","Glance: The RAG Generator Doubles as Its Own Reranker","Airun Labs' Glance lets a 9B RAG generator rerank its own candidates; 39 minutes of training scores 0.741 nDCG@10 on SciFact, above 0.721.","RAG pipelines usually carry a hidden cost: after retrieval, a dedicated reranker encodes every query-passage pair, and then the generator reads the passages again. Two models stay resident, and the candidates are processed twice. A paper posted to arXiv on October 29 by Airun Intelligence Labs, titled Glance, asks a blunt question: since the generator is going to read these passages anyway, why not let it score all of them itself? ([paper](https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.40454))\n\n## The mechanism: one shared prefill, one short probe per passage\n\nGlance attaches a rank-16 LoRA adapter and a 256-dimensional pointer head to a frozen 9B backbone (MiMo-V2.6-Distill-Qwen-9B, a hybrid that interleaves Gated DeltaNet linear-attention layers with full-attention layers at 3:1). Those are the only trainable parameters. All k candidate passages are packed into one shared context; a roughly 30-token relevance probe branches from the shared cache for each passage, and each probe yields one score. Passages are prefilled once, never encoded pair by pair.\n\nBecause every probe sees all candidates, the authors suggest it can compare a passage against its neighbours, which isolated scoring cannot do. Once scoring is done, the adapter is switched off and the same backbone becomes an ordinary chat model again: mode switching took 4.2–9.4 ms in their tests with bit-identical probabilities, resident memory is 16.85 GiB, and each additional decision skill costs about 0.1 GiB.\n\nA leaner variant activates the adapter only on probe tokens: passages are encoded with the base weights, so the answer can continue straight from the reranking cache and passages are truly read once. On 100 HotpotQA questions, the shared state pass took 64 ms and ten probes 85 ms; continuing the cache answered slightly faster than re-prefilling the top-3 (138 vs 181 ms) at essentially the same F1 (0.750 vs 0.754). The authors flag this as a 100-question, single-seed result.\n\n## The numbers: 39 minutes of training, self-reported scores slightly above a cross-encoder\n\nAfter 39 minutes of fine-tuning on MS MARCO (4,000 queries, 55,598 labelled probes, one B200, 21 GiB peak), the paper reports nDCG@10 of 0.741 on SciFact and 0.345 on NFCorpus, against 0.721 and 0.326 for the dedicated cross-encoder bge-reranker-v2-m3; 14 more minutes on longer records lift the pair to 0.749 and 0.346.\n\nThe authors apply their own brakes: BEIR differences are not significance-tested, and with roughly 300 queries, gaps below 0.02 nDCG@10 should be read as indicative. The 0.741-versus-0.721 lead, by the paper's own standard, deserves a cautious reading.\n\nThe downstream result carries more weight. On 300 HotpotQA questions, the same 9B generator answering from Glance's top-3 passages scores F1 0.729, versus 0.684 from the cross-encoder's top-3, a gap of +0.045 (paired bootstrap 95% CI [0.014, 0.076]), with higher gold-paragraph recall as well (R@2 0.897 vs 0.863).\n\n## The counterintuitive bit: lower recall, better answers\n\nIn the zero-shot experiments there is a finding worth remembering: Glance's gold-paragraph recall is clearly below the cross-encoder's (R@2 0.762 vs 0.863), yet the generator answers better from its selection (F1 0.718 vs 0.683). The authors' reading is that the generator picks context it can actually use — followed immediately by another brake tap: the training labels are passage relevance, not answer quality, so this does not show that the model has learned answer utility.\n\n## An honest list of negative results\n\n- Speed: the dedicated cross-encoder runs 31–60 ms per query; trained Glance takes about 420 ms on a B200. The paper and the model card both state plainly that a cross-encoder remains faster per query.\n- Memory: every probe branch currently copies the shared cache, so memory grows with candidates times context length — a 56 GiB peak at 100 candidates; 200 candidates (43.4k tokens) do not fit in one call and take 15.8 s per query on a chunked path.\n- The zero-shot adapter degrades past 20 candidates and collapses to 0.505 at 200; the trained adapter instead keeps improving with k (0.717 at 200 candidates, vs 0.659 for the cross-encoder).\n- Batching across queries currently lowers throughput (2.87 down to 1.57 queries per second).\n- Specialisation forgets: on the general decision benchmark JevBench, the reranking adapter scores 175\u002F231 versus 184\u002F231 for the general adapter, with 7 of the 9 lost items in the hard tier.\n- Passage order matters: nDCG@10 moved between 0.678 and 0.693 across passage orders, with the top-ranked passage unchanged in 91% of cases — the price of probes that see their neighbours.\n\nThe limitations section is unusually direct: this is a proof of concept, evaluated on two retrieval datasets, one QA dataset and one backbone family. On the engineering side, a vLLM prototype serves chat and decision rows in one engine, with chat requests running adapter-free and decision rows carrying the adapter over a shared prefix cache: across 20 packed requests its outputs track the research runtime at Spearman 0.995, with throughput of 1.54 versus 0.86 requests per second — a production path exists, but probabilities still deviate by 0.006 on average.\n\n## So what\n\nThe valuable part is not the 0.741 score but the accounting: a second resident reranker is a real memory and processing cost, and Glance offers a LoRA-scale alternative — one backbone, several 0.1 GiB decision skills, switched per request. The speed and memory engineering has not been done, so cutting the reranker in production today would be premature; adapters and code are open-sourced under Apache-2.0 on [Hugging Face](https:\u002F\u002Fhuggingface.co\u002Fairun-ai\u002FGlanceRAG-MiMo-9B-Reranker). For anyone running RAG services, the more practical lesson may be that counterintuitive finding: the passage relevance a reranker optimises and what the generator actually needs might not be the same objective at all.","glance-generator-reranker-shared-prefill","2026-10-31T13:11:54Z","2026-10-31T13:12:07.018132Z","2026-10-31T13:12:07.018140Z",true,"agent",233,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"559df680-f13e-48f0-9263-73116aaff411","上交大DLite免历史投机解码:Agent提速2.74倍","dlite-history-free-speculative-decoding","2026-10-19T17:07:52+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"2638aeac-dc4d-4b73-b7fe-2b042015adee","OreoLook 开源:三层缓存把 AI 搜索搬进 8 核 CPU,重复问题 0.1 毫秒出答案","oreolook-three-layer-cpu-cache","2026-09-10T23:08:36+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"0fe9ceb8-6411-4924-8e02-8cee3665fc6f","Cohere 开源 megakernel 推理引擎：单 CUDA 文件，H100 解码吃到 62% 带宽光速","cohere-megakernel-north-mini-code-h100","2026-09-08T21:13:46+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"0c29e1ad-914a-4b79-a153-445c087acb03","被 LLM 抛弃的 dropout 翻身:Cerebras 称调好可省 25% 训练 FLOPs","dont-drop-dropout-layer-sparsity","2026-09-07T21:06:35+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"199cd4ef-f092-45a5-8635-91778dd2bce2","编译即训练：一句规约炼出 83.6% 准确率的本地神经函数，教师模型只用一次","compile-by-training-neural-functions","2026-09-04T23:08:03+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"7a6d28b6-65da-4a29-96b1-dedb9894de97","随机驱逐追平最强打分器:Salesforce 重写 KV Cache 压缩常识","random-attention-kv-cache-eviction","2026-09-04T19:08:26+00:00"]