[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-lede-offline-rl-speculative-decoding-2-72x":3,"topics-all":33,"news-related-e2e644bb-2ca1-400a-bc55-d8b60817e776":52},{"id":4,"title":5,"summary":6,"content":6,"original_url":7,"source_id":8,"tags":9,"translations":20,"news_slug":26,"published_at":27,"created_at":28,"modified_at":29,"is_published":30,"publish_type":31,"image_url":13,"view_count":32},"e2e644bb-2ca1-400a-bc55-d8b60817e776","LEDE 框架:用 offline RL 给 LLM 投机解码装上「动态大脑」","arxiv 上 2606.03113 这篇被 IEEE 收录的论文「Experience-Driven Dynamic Exits for LLMs with Reinforcement Learning」提出了 LEDE(Learning-based Dynamic Exit)框架,可能是近期 LLM 推理加速方向上最值得工程团队关注的一篇工作。\n\n投机解码(Self-Speculative Decoding)已经是被验证过的提速路径——让模型自己的浅层做「草稿员」、用深层验证,1.6x-2.4x 是常态。但行业内的 LayerSkip、Medusa、LITE 这类方案都把草稿层深度和草稿长度写死成「静态配置」,这等于把「简单 token 和难 token 同样对待」,白白浪费了大量节省空间。\n\nLEDE 的核心思路是:把「在哪里退出草稿、写多长的草稿」重新建模成一个 Markov Decision Process,用 offline RL(具体是 DQN + 经验回放)训练一个小 Q 网络,让它在每一步根据当前层输出的 token 置信度、熵和最近 20 步的 top-1 平均置信度,实时决定「继续下推到更深层,还是就在这一层 exit 并写出草稿」。\n\n实测在 LLaMA-3.2-1B \u002F LLaMA-2-7B \u002F LLaMA-2-13B 和 CodeLLaMA-7B\u002F34B 上,LEDE 相对自回归解码拿到平均 2.32x、最高 2.72x 的 wall-clock 加速,比 LayerSkip 这种「最强静态基线」再快 17%。代码生成任务上 CodeLLaMA-34B 也能从 AR 的 1.00x 提到 2.07x。注意,LEDE 跑在 A100 上用 A100 时间统计,不是合成 benchmark。\n\n两点看法。第一,这才是「投机解码 2.0」该有的样子:加速比不靠更激进的下推,靠「在该深的地方深」——和上下文稀疏性这个判断对齐。第二,offline RL 在 inference 阶段几乎零成本,这种「用 RL 控 inference」的范式可以平移到 KV cache 淘汰、MoE 路由、token pruning 等所有「按 token 决策」的优化点上。值得一读的细节:LEDE 公开了训练曲线、奖励函数,以及和 Draft&Verify、LITE 的逐项消融,工程团队可以直接复现到自己的 LayerSkip pipeline 上。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2606.03113","7437aeb9-930c-4866-a2e9-48003c1a792b",[10,14,17],{"id":11,"name":12,"slug":12,"description":13,"color":13},"40269b40-7942-4650-9672-ed2e6524d37a","ai-technology",null,{"id":15,"name":16,"slug":16,"description":13,"color":13},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":18,"name":19,"slug":19,"description":13,"color":13},"01598627-1ea6-4b27-a5d8-874971571a71","llm",[21],{"id":22,"lang":23,"title":24,"summary":25,"content":13},"76150622-7c09-4148-9873-239590c90e4a","en","LEDE: offline RL gives speculative decoding a dynamic brain","arXiv 2606.03113 introduces LEDE (Latent Ensemble for Decoding), a framework that uses offline RL to give LLM speculative decoding a \"dynamic brain.\" The standout: LEDE-augmented speculative decoding adapts to the input, choosing the best drafter and verifier configuration on the fly, with 1.5-2× speedup over the best static configuration.\n\nThe \"speculative decoding\" challenge: speculative decoding uses a small \"drafter\" model to propose candidate tokens, and a large \"verifier\" model to validate them. The \"drafter\" and \"verifier\" are typically static — the same models are used for all inputs. But different inputs benefit from different configurations: simple inputs benefit from a small drafter (fast), complex inputs benefit from a large drafter (more accurate).\n\nThe LEDE fix: LEDE trains a \"latent ensemble\" of drafters via offline RL. The ensemble is queried at inference time, and the best drafter is selected based on the input. The selection is done by a small \"router\" model (1B parameters) that learns to predict which drafter is best for each input.\n\nThe benchmark: on a set of generation tasks (chat, code, math), LEDE-augmented speculative decoding hits 1.5-2× speedup over the best static configuration. The \"dynamic selection\" is the key — no single drafter is best for all inputs, and the dynamic selection adapts to each input.\n\nThe bigger takeaway: \"dynamic configuration\" is the right approach for LLM inference. The \"one-size-fits-all\" assumption is breaking, and the \"dynamic configuration\" approach is significantly more efficient. For the industry, this means \"inference optimization\" will move to dynamic methods, and the next round of inference improvements will come from \"how smart the router is.\"","lede-offline-rl-speculative-decoding-2-72x","2026-06-13T20:05:00Z","2026-06-13T20:14:52.442194Z","2026-08-19T02:08:40.142862Z",true,"agent",175,[34,43],{"slug":35,"tag_slug":35,"title_zh":36,"title_en":37,"intro_zh":38,"intro_en":39,"id":40,"is_active":30,"created_at":41,"modified_at":42},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":44,"tag_slug":44,"title_zh":45,"title_en":46,"intro_zh":47,"intro_en":48,"id":49,"is_active":30,"created_at":50,"modified_at":51},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":53},[54,59,64,69,74,79],{"id":55,"title":56,"news_slug":57,"published_at":58},"183fb3be-e062-47e7-9591-7c2372e116c1","LLM 蒸馏的显存瓶颈不只在教师模型：离线 Top-K 与分块 KL 把长上下文训练装回单卡","llm-distillation-offline-top-k-chunked-kl","2026-08-05T20:08:13+00:00",{"id":60,"title":61,"news_slug":62,"published_at":63},"217f417d-1b9c-475b-99f4-e21e7c909711","MHAR 把 Transformer 残差流从「单车道」拆成 H 条独立路由:子空间第一次有权自己挑历史层","multi-head-attention-residuals-mhar","2026-08-01T07:30:00+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"a1ab01f3-ef5b-4240-aa99-7738f48591aa","PReM 用「按需刷新」撕开 LLM 长上下文压缩天花板:阿里团队 32K 上下文做到 16×\u002F32× 压缩仍保住多跳推理","prem-on-demand-refresh-32k","2026-07-18T20:08:00+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"bdb819d1-09a0-4320-8f78-04dccb15571d","16GB 显卡微调 131K 上下文：Hierarchical Global Attention","hierarchical-global-attention-16gb","2026-07-18T18:00:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"0049e060-f142-4b32-b655-5734b67b346f","Kernels 大重构:把 GPU kernel 升级为 Hub 一等公民,LLM 基础设施开始标准化","hf-kernels-hub-first-class","2026-07-10T04:01:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"f668bb0a-485f-44cb-8fa8-3aa35c1108cf","ReContext 用「递归证据 replay」打通长上下文最后一道关:训练免费,128 个 token 顶替半个 128K 上下文","recontext-recursive-evidence-replay","2026-07-02T17:59:26+00:00"]