[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-real-loss-unbiased-eal-draft-training":3,"topics-all":35,"news-related-a69f25d4-8921-4419-a9a4-e074378cb15e":64},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":21,"news_slug":28,"published_at":29,"created_at":30,"modified_at":31,"is_published":32,"publish_type":33,"image_url":14,"view_count":34},"a69f25d4-8921-4419-a9a4-e074378cb15e","港大实数版EAL:投机解码训练目标的梯度会反号","港大单作者论文 REAL 指出,投机解码的 block 级训练目标把「先相乘再平均」写成「先平均再相乘」,丢掉接受分支的协方差,梯度方向都可能反转。改用路径权重构造无偏 EAL 估计器后,Qwen3.6-35B-A3B 上概率 serving 接受长度 4.792 超过全部基线,且只改损失函数。","训练投机解码的草稿模型,最常用的损失是交叉熵和 KL 散度。可推理时真正决定加速比的,是每轮平均能接受几个草稿 token——期望接受长度 EAL。港大 Yu Tian 的单作者论文 REAL 把这层错位拆到了底:不止传统目标在优化别的东西,连近期专门冲着 EAL 设计的 block 级目标,也带着一个能把梯度方向整个反过来的系统偏差。\n\n## 乘法顺序错了,梯度跟着反\n\nWTV 这类 block 级目标把每步接受率相乘,去近似整窗的存活概率。问题在被接受的 token 有双重身份:它既延长已接受前缀,又决定下一个草稿 token 在什么前缀上接受验证。「先平均再相乘」丢掉了这层分支结构,精确目标应该给每条路径乘上权重再平均——两者之差恰好是路径权重与后续接受率的协方差。论文的反例干净利落:取一个两步、两个词表项的迷你设定,第一步目标分布是 0.4 对 0.6,草稿分布是 0.8 对 0.2,第二步的目标分布随第一步接住的分支而变;直接枚举四条路径后,精确目标的梯度是正的、奖励调大草稿参数,代理目标的梯度是负的、奖励调小,在 θ=0.5 处两者数值相等、方向相反。这不是玩具担忧——作者还记录了训练全程的协方差:始终为正,意味着代理在系统性低估真目标。\n\n## REAL:无偏估计器,只改损失函数\n\nREAL 的做法是在冻结 target 采样的轨迹上,给每个 token 乘路径权重,恢复提出即接受的分支分布,得到 EAL 的无偏估计;再用 Rao–Blackwellization 把每项末端的采样因子换成条件期望,第 2 个位置的单项梯度方差降 76–84%,聚合梯度方差降 52–58%;外加一项辅助交叉熵稳定从头训练,消融显示纯交叉熵和去掉交叉熵的变体都更差,两个成分各贡献一半。改动全部落在损失函数上——草稿器架构、训练管线、推理流程一概不动,也不引入额外训练开销。\n\n## 4.792 的成绩单和它的边界\n\n实验冻结 Qwen3.6-35B-A3B(MoE 与 GDN 混合架构,贴近当前主流模型的设计选择)当 target,配 Domino 草稿器,每轮 1 个锚点加 γ=7 的草稿窗口,3 个 epoch 共 140K 步,数据用 OpenPerfectBlend 提示词由 target 在温度 1.0 重新生成。评测在 vLLM 引擎上跑 9 个任务子集、5 个随机种子,区分概率采样和 greedy 两种 serving 协议。概率采样 serving 下,REAL 的接受长度 4.792,去掉 Rao–Blackwellization 的变体 4.791,压过 static LK 的 4.740、WTV 的 4.715、VSD 的 4.657、DBLast 的 4.749,九个子集的最优全部由这两个变体包揽;第 1 个位置各家几乎打平,差距是在后面位置拉开的——第 7 个位置的接受保留率 0.315,高于 LK 的 0.306。优势还会随训练累积:对 WTV 的领先从第 1 个 epoch 后的 0.041 扩到第 3 个的 0.077,续训到第 4 个 epoch,REAL 到 4.816,没有饱和迹象。\n\n两个边界必须写明:greedy serving 下最高的其实是 static LK(4.544 对 REAL 的 4.537);把采样温度降到 0.5,static LK 以 5.002 反超 REAL 的 4.993。REAL 的优势在随机采样 serving,越接近确定性解码越窄——这是论文自己声明的适用范围,不是遮掩。验证也只覆盖单一 target、单一草稿器、单一数据配方;VSD 和 DBLast 的对比是损失级改编而非完整系统复现。\n\n## 所以呢\n\n这篇的真正贡献不是零点零几的接受长度,而是方法论警告:当训练指标与真实目标之间隔着代理,先问一句「代理无偏吗」。作者实测常用的训练期 EAL 代理指标全程偏向 WTV,与真实 serving 排序完全相反——因为这个代理和 WTV 共享同一套乘积形式,指标在悄悄骗人。工程上这是个即插即用的损失函数,参考实现挂在 vLLM speculators 库的 fork 上([King-ty\u002Fspeculators](https:\u002F\u002Fgithub.com\u002FKing-ty\u002Fspeculators\u002Ftree\u002Freal-loss));同期还有 EDR、BV loss 等工作从动态规划、块验证角度攻同一问题,这个方向正在变热。对把投机解码当推理降本主路径的团队,目标函数级的修正比再堆一层系统技巧更值得跟进。当然,单 target、单草稿器的验证规模也摆在那里([论文](https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.14399)),结论外推前值得多看几家。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.14399","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18],{"id":12,"name":13,"slug":13,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":19,"name":20,"slug":20,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",[22],{"id":23,"lang":24,"title":25,"summary":26,"content":27},"db5e58c5-e469-43d6-9263-e60112b5d664","en","REAL loss: unbiased EAL training for speculative decoding","HKU's REAL loss fixes gradient-reversing draft-training surrogates, hitting 4.792 acceptance on Qwen3.6-35B-A3B by changing only the training loss.","Training a draft model for speculative decoding usually means cross-entropy or KL divergence. But what actually governs the speedup at inference time is the expected acceptance length (EAL) — how many draft tokens survive verification per round on average. A single-author paper from the University of Hong Kong, REAL, takes this misalignment apart: not only do conventional objectives optimize something else entirely, even recent block-level objectives designed specifically for EAL carry a systematic bias that can flip the gradient direction.\n\n## Multiply Before You Average\n\nBlock-level objectives such as WTV multiply per-step acceptance rates to approximate the survival probability over the window. The problem is that an accepted token has a double identity: it extends the accepted prefix, and it determines the prefix on which the next draft token gets verified. Averaging first and multiplying afterward discards this branch structure, while the exact objective weights each path before averaging — and the difference is exactly the covariance between path weights and downstream acceptance rates. The paper's counterexample is crisp: in a two-step setting with a two-token vocabulary, the first-step target distribution is 0.4 versus 0.6 and the draft distribution is 0.8 versus 0.2, with the second-step target depending on which branch survives. Enumerating all four paths directly, the exact objective's gradient is positive and rewards raising the draft parameter, while the surrogate's is negative and rewards lowering it; at θ=0.5 the two objectives take equal value with opposite gradients. Nor is this a toy concern — the author logs this covariance throughout training: it stays positive, meaning the surrogate systematically underestimates the true objective.\n\n## REAL: An Unbiased Estimator, Loss-Only Change\n\nREAL samples trajectories from the frozen target, multiplies each token by a path weight to recover the propose-and-accept branch distribution, and obtains an unbiased estimator of EAL. Rao–Blackwellization then replaces the final sampled factor of each term with its conditional expectation, cutting per-term gradient variance by 76–84% at position 2 and aggregate gradient variance by 52–58%; an auxiliary cross-entropy term stabilizes training from scratch. Ablations show pure cross-entropy and the CE-free variant both fall short — each component contributes. Every change lives in the loss function: drafter architecture, training pipeline, and inference procedure all stay untouched, with no additional training overhead.\n\n## The 4.792 Scorecard and Its Edges\n\nThe experiments freeze Qwen3.6-35B-A3B (a MoE model with a hybrid GDN structure, chosen to mirror current mainstream architecture) as the target, pair it with a Domino drafter, run one anchor plus a γ=7 draft window per round, and train for 3 epochs totaling 140K steps on OpenPerfectBlend prompts regenerated by the target at temperature 1.0. Evaluation runs on the vLLM engine across 9 task subsets and 5 random seeds, under both probabilistic and greedy serving protocols. Under probabilistic serving, REAL reaches 4.792 acceptance length, with the variant dropping Rao–Blackwellization at 4.791 — ahead of static LK at 4.740, WTV at 4.715, VSD at 4.657, and DBLast at 4.749. The best score on every one of the nine subsets belongs to one of the two REAL variants. At position 1 everyone is nearly tied; the gap opens later — acceptance retention at position 7 is 0.315 versus 0.306 for static LK. The advantage also accumulates: the lead over WTV grows from 0.041 after the first epoch to 0.077 by the third, and continuing into a fourth epoch REAL reaches 4.816 with no sign of saturation.\n\nTwo boundaries must be stated. Under greedy serving, static LK actually tops the table at 4.544 against REAL's 4.537; and when the sampling temperature drops to 0.5, static LK overtakes REAL 5.002 to 4.993. REAL's edge lives in stochastic sampling — the closer decoding gets to deterministic, the narrower it becomes. That is the paper's own stated scope, not a defect being hidden. Validation also covers a single target, a single drafter, and a single data recipe; the VSD and DBLast comparisons are loss-level adaptations rather than full system reproductions.\n\n## So What\n\nThe real contribution here is not a few hundredths of acceptance length but a methodological warning: whenever a training metric stands in for the true objective, ask first whether the proxy is unbiased. The author measured the standard training-time EAL proxy favoring WTV throughout training while the true serving ranking says the opposite — unsurprising, since that proxy shares the same product-of-marginals form as WTV; the metric was quietly lying. In engineering terms this is a drop-in loss, with a reference implementation on a fork of the vLLM speculators library ([King-ty\u002Fspeculators](https:\u002F\u002Fgithub.com\u002FKing-ty\u002Fspeculators\u002Ftree\u002Freal-loss)). Concurrent efforts such as EDR and BV loss attack the same problem from dynamic-programming and block-verification angles — the direction is heating up. For teams treating speculative decoding as their main inference cost lever, a correction at the objective level deserves attention before yet another layer of systems tricks. That said, the validation covers a single target and a single drafter ([paper](https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.14399)) — extrapolate with care.","real-loss-unbiased-eal-draft-training","2026-11-12T18:05:00Z","2026-11-12T17:33:16.442727Z","2026-11-12T17:33:16.442738Z",true,"agent",134,[36,46,55],{"slug":37,"tag_slug":38,"title_zh":39,"title_en":40,"intro_zh":41,"intro_en":42,"id":43,"is_active":32,"created_at":44,"modified_at":45},"image-generation","text-to-image","图像生成：从文生图到逐框编辑的模型竞赛","Image Generation: from text-to-image to frame-level editing","文生图已是红海，战场正转向编辑与可控性：FLUX.2\u002F3 用开源和量化把生图搬进消费级显卡，Qwen-Image、Seedream 等国产模型紧咬不放，GPT Image 2 则把推理能力带进像素。本专题追踪图像生成与编辑模型的发布、架构演进、加速量化与商业化数据，看「好图」如何变成「可用的图」。","Text-to-image is a crowded race, and the frontier has moved to editing and control: FLUX.2\u002F3 push open weights and quantization onto consumer GPUs, Qwen-Image and Seedream close in from China, and GPT Image 2 brings reasoning into pixels. This topic tracks model releases, architecture shifts, acceleration work, and the business numbers behind \"pretty\" becoming \"usable\".","52de7c59-7864-4569-8ae7-3d656d26a1a1","2026-11-20T07:20:23.534506Z","2026-11-20T07:20:23.534527Z",{"slug":47,"tag_slug":47,"title_zh":48,"title_en":49,"intro_zh":50,"intro_en":51,"id":52,"is_active":32,"created_at":53,"modified_at":54},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":56,"tag_slug":56,"title_zh":57,"title_en":58,"intro_zh":59,"intro_en":60,"id":61,"is_active":32,"created_at":62,"modified_at":63},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":65},[66,71,76,81,86,91],{"id":67,"title":68,"news_slug":69,"published_at":70},"abd8eb9f-c106-4899-bf24-b6bbed3da5e3","EPFL论文实测:注意力砍维度而非砍token,只算三成点积","hisa-hidden-dimension-sparse-attention-rope","2026-11-20T09:00:00+00:00",{"id":72,"title":73,"news_slug":74,"published_at":75},"4544ed60-b5fb-41b6-9fee-dd2f59012547","隐状态转个方向:直答46到85,超CoT省九成token","cot-hidden-state-directions-direct-answers","2026-11-19T13:30:00+00:00",{"id":77,"title":78,"news_slug":79,"published_at":80},"9af27b13-b1a9-47e3-a143-fd8a6c1ee5a9","Gemma 4省下的37.5%,推理引擎还没兑现","gemma-4-kv-cache-k-equals-v-compact-row","2026-11-18T11:40:00+00:00",{"id":82,"title":83,"news_slug":84,"published_at":85},"76eb5d5a-04c1-4d7f-9c50-dc494b1c3bb8","扩散LM宽窗塌方解剖:一成回退提交,点燃七成失败","diffusion-lm-fallback-commit-failure-roots","2026-11-17T15:35:00+00:00",{"id":87,"title":88,"news_slug":89,"published_at":90},"09df4ac8-eede-4a9c-b488-0a577b589417","0.6B语音模型不生成答案,只回答是非题","marimba-typed-speech-decision-frozen-asr","2026-11-17T01:00:00+00:00",{"id":92,"title":93,"news_slug":94,"published_at":95},"d9243227-680a-4fde-908d-8990969a54e0","LuthorSQL：7B 模型替 Agent 省三成账单","luthorsql-text-to-sql-agent-cascade","2026-11-16T05:00:00+00:00"]