[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-dlite-history-free-speculative-decoding":3,"topics-all":38,"news-related-559df680-f13e-48f0-9263-73116aaff411":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"559df680-f13e-48f0-9263-73116aaff411","上交大DLite免历史投机解码:Agent提速2.74倍","上海交大发布投机解码框架 DLite,专攻 Agent 负载。论文指出草稿模型在长上下文多轮交互下接受长度最多衰退 80%,根因是历史上下文膨胀;DLite 复用目标模型隐状态做局部预测,不维护草稿历史。SWE-Bench 上配 Qwen3-8B 加速 2.74 倍,比 DSpark 高约 71%,已开源。","Agent 应用把大模型推理拖进了一个尴尬区间:上下文越滚越长、多轮交互越来越密,而主流加速手段——投机解码——恰恰在这种负载下失灵。上海交大团队发布的 DLite 框架(arXiv:2610.23787)给出了一份诊断:问题不在草稿模型不够强,而在它背着的历史包袱。\n\n## 两个问题,一个根因\n\n投机解码的常规做法,是让一个轻量草稿模型先生成候选 token,再由目标模型批量验证。论文观察到,在 Agent 工作负载下,现有草稿模型的接受长度最多衰退 80%;与此同时,草稿模型自己维护一份历史 KV cache,也带来可观的系统开销。\n\n作者把两个问题归结到同一个根源:不断膨胀的历史上下文。上下文越长,草稿模型要建模的依赖关系越复杂,预测质量随之下滑;缓存也随轮次持续增长,两头一起恶化。\n\n## DLite 的解法:不维护历史,直接复用\n\nDLite 的思路是彻底绕开历史上下文:直接复用目标模型的上下文隐状态,在局部做投机预测,不再显式维护草稿侧的历史信息,把一个越滚越复杂的序列建模问题降维成局部预测问题。块内的因果依赖交给一个轻量序列头,以低开销恢复;训练侧配套两阶段策略,对齐这个异构的无历史架构。\n\n## 跑分:SWE-Bench 2.74 倍,压过 DSpark\n\n按论文 Table 2\u002F3(硬件 NVIDIA H800,目标模型 Qwen3-4B\u002FQwen3-8B,投机块大小 K=7):\n\n- Agent 负载:端到端加速 2.07–2.74 倍;SWE-Bench 上 Qwen3-8B 达 2.74 倍,对比 DSpark 的 1.60 倍,高出约 71%\n- 长上下文(LongBench v2):多文档问答、长程上下文学习、代码库理解、长对话历史四类场景 2.18–3.96 倍\n- 通用负载:数学\u002F代码\u002F聊天基准平均接受长度 3.64–6.48 token\n- 演示中 46,704 token 的长输入也能保持加速\n\n值得点出的是,8 月 Liquid AI 的 LFM2.5-DSpark 曾以 300M 草稿模型把端侧投机解码推到 2.87 倍;两个月后,上交大在 Agent 场景把这条 SOTA 线又抬了一截——这条细分赛道的迭代速度可见一斑。\n\n## 开源与限制\n\n框架以 MIT 协议开源,GitHub 提供 Python API、CLI 和吞吐基准,配套训练评测框架 LiteSpec 还打包了 DFlash、DSpark 等对比方法;Qwen3-4B\u002F8B\u002F14B 与 Qwen3.5-4B 四档草稿 checkpoint 权重均已放出。\n\n但 README 的路线图同样诚实:SGLang、vLLM 集成还在待办清单上,更大目标模型的支持也未落地。想在生产 serving 栈里直接吃到这 2.74 倍,还得等工程化跟进——这是评估现在能不能用的关键边界。\n\n回到更根本的问题:当 Agent 把上下文滚到几万 token,任何为单轮对话设计的加速方案都得重新答卷。DLite 的答案是干脆把历史包袱扔掉——草稿模型不再自己记历史,而是搭目标模型的便车。这个方向值得所有做推理优化的人盯住。\n\n参考:arXiv:2610.23787(https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.23787);代码 github.com\u002FSJTU-DAI-Sys\u002FDLite","https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.23787","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":19,"name":20,"slug":20,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":22,"name":23,"slug":23,"description":14,"color":14},"b9bd9039-fcdb-41a8-b85b-fc1587def2b9","open-source",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"71e5957c-1fd1-4225-b755-b7d6b1ed4041","en","DLite: History-Free Speculative Decoding Hits 2.74x on Agents","SJTU's DLite removes history maintenance from agent speculative decoding, hitting 2.74x on SWE-Bench and 3.96x on LongBench, with open weights.","Agentic workloads have dragged LLM inference into an awkward zone: contexts keep growing, multi-turn interactions pile up, and the mainstream acceleration technique — speculative decoding — loses much of its power exactly there. A Shanghai Jiao Tong University team now offers a diagnosis in their DLite framework (arXiv:2610.23787): the problem is not that draft models are too weak, but that they carry a growing historical burden.\n\n## Two Problems, One Root Cause\n\nStandard speculative decoding lets a lightweight draft model generate candidate tokens that the target model verifies in batch. The paper observes that under agentic workloads, existing draft models suffer up to an 80% degradation in acceptance length, while the draft model's own historical KV cache adds substantial system overhead.\n\nThe authors trace both problems to a single root: the ever-growing historical context. Longer contexts force the draft model to model increasingly complex dependencies, dragging prediction quality down, while the cache keeps expanding turn after turn.\n\n## The DLite Approach: Drop History, Reuse Hidden States\n\nDLite sidesteps historical context entirely. It directly reuses contextual hidden states from the target model and performs local speculative prediction without maintaining any history on the draft side — turning an increasingly complex sequence-modeling problem into a local prediction task. A lightweight sequential head restores intra-block causal dependencies at low overhead, and a tailored two-stage training strategy aligns this heterogeneous history-free architecture.\n\n## Numbers: 2.74x on SWE-Bench, Beating DSpark\n\nFrom Tables 2 and 3 of the paper (NVIDIA H800 GPUs, Qwen3-4B \u002F Qwen3-8B targets, speculative block size K=7):\n\n- Agentic workloads: 2.07–2.74x end-to-end speedup; on SWE-Bench with Qwen3-8B it reaches 2.74x versus DSpark's 1.60x — about 71% higher\n- Long context (LongBench v2): 2.18–3.96x across multi-document QA, long in-context learning, code repository understanding, and long dialogue history\n- General workloads: average acceptance length of 3.64–6.48 tokens across math, coding, and chat benchmarks\n- A demo keeps the speedup on a 46,704-token long input\n\nNotably, Liquid AI's LFM2.5-DSpark pushed edge-side speculative decoding to 2.87x with a 300M draft model back in August; two months later, the SJTU team has lifted that SOTA line again in agentic settings — the pace of iteration in this niche is striking.\n\n## Open Source, With Honest Limits\n\nThe framework is open-sourced under MIT, with a Python API, CLI, and throughput benchmark on GitHub; the companion LiteSpec training and evaluation framework also packages DFlash and DSpark baselines. Draft checkpoints for Qwen3-4B\u002F8B\u002F14B and Qwen3.5-4B are all released.\n\nThe README's roadmap is equally honest, though: SGLang and vLLM integrations are still on the to-do list, and support for larger target models has not landed. Eating that 2.74x in a production serving stack will have to wait for engineering follow-through — the key boundary when judging whether this is usable today.\n\nThe deeper question stands: when agents roll contexts to tens of thousands of tokens, every acceleration scheme designed for single-turn chat must re-sit its exam. DLite's answer is to throw the historical burden away — the draft model no longer keeps its own history, it hitches a ride on the target model. Anyone working on inference optimization should watch this direction.\n\nReference: arXiv:2610.23787 (https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.23787); code at github.com\u002FSJTU-DAI-Sys\u002FDLite","dlite-history-free-speculative-decoding","2026-10-19T17:07:52Z","2026-10-19T17:08:18.310340Z","2026-10-19T17:08:18.310353Z",true,"agent",121,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"2638aeac-dc4d-4b73-b7fe-2b042015adee","OreoLook 开源:三层缓存把 AI 搜索搬进 8 核 CPU,重复问题 0.1 毫秒出答案","oreolook-three-layer-cpu-cache","2026-09-10T23:08:36+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"0fe9ceb8-6411-4924-8e02-8cee3665fc6f","Cohere 开源 megakernel 推理引擎：单 CUDA 文件，H100 解码吃到 62% 带宽光速","cohere-megakernel-north-mini-code-h100","2026-09-08T21:13:46+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"0c29e1ad-914a-4b79-a153-445c087acb03","被 LLM 抛弃的 dropout 翻身:Cerebras 称调好可省 25% 训练 FLOPs","dont-drop-dropout-layer-sparsity","2026-09-07T21:06:35+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"199cd4ef-f092-45a5-8635-91778dd2bce2","编译即训练：一句规约炼出 83.6% 准确率的本地神经函数，教师模型只用一次","compile-by-training-neural-functions","2026-09-04T23:08:03+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"7a6d28b6-65da-4a29-96b1-dedb9894de97","随机驱逐追平最强打分器:Salesforce 重写 KV Cache 压缩常识","random-attention-kv-cache-eviction","2026-09-04T19:08:26+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"44a035c8-b8a3-48e5-af4f-c76323dac7b5","RWKV7-G1j 13.3B 开源:不用注意力,每 token 推理成本是常数","rwkv7-g1j-13b-attention-free","2026-09-03T13:14:19+00:00"]