[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-saguaro-ssd-5x-speculative-speculative-decoding":3,"topics-all":33,"news-related-bba689fe-155f-4ff1-a21b-6791b4fb3651":52},{"id":4,"title":5,"summary":6,"content":6,"original_url":7,"source_id":8,"tags":9,"translations":20,"news_slug":26,"published_at":27,"created_at":28,"modified_at":29,"is_published":30,"publish_type":31,"image_url":13,"view_count":32},"bba689fe-155f-4ff1-a21b-6791b4fb3651","投机解码再进化：Saguaro实现LLM推理速度5倍提升","自回归解码的顺序性瓶颈正在被打破。5月1日，一篇发表于OpenReview的论文提出了「投机双重解码」（Speculative Speculative Decoding，SSD）算法，在投机解码的基础上进一步并行化预测流程，实现最高5倍的速度提升。\n\n传统投机解码使用快速draft模型预测后续token，再由主模型并行验证。但预测与验证之间仍存在顺序依赖——draft模型必须等当前验证完成才能开始下一次预测。\n\nSSD的核心思路是让draft模型在验证进行时就「猜测」验证结果并预先准备候选。当实际验证命中预测时，可立即返回结果，消除drafting延迟。研究者将算法实现为Saguaro，在开源推理引擎上测试，相比优化后的投机解码基线加速2倍，相比纯自回归解码加速5倍。\n\n这项工作的价值在于它不是另一个花哨的工程技巧，而是从算法层面重新审视了投机解码的瓶颈结构。它指出真正的问题不是「draft模型能不能预测」，而是「draft模型等待验证时在做什么」——答案是什么都没做，所以要让它同时做更多。\n\n当然，5倍的提升数字来自开源引擎上的特定场景，实际部署收益会因模型规模、硬件配置和请求类型而有所不同。但它指向的方向是明确的：LLM推理的加速空间还没有被穷尽，而下一个突破口可能不在模型本身，而在调度层的重新设计。","https:\u002F\u002Fopenreview.net\u002Fforum?id=aL1Wnml9Ef","ec0a79b7-694c-4caf-8071-91315d69c706",[10,14,17],{"id":11,"name":12,"slug":12,"description":13,"color":13},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",null,{"id":15,"name":16,"slug":16,"description":13,"color":13},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":18,"name":19,"slug":19,"description":13,"color":13},"01598627-1ea6-4b27-a5d8-874971571a71","llm",[21],{"id":22,"lang":23,"title":24,"summary":25,"content":13},"784a87d3-7d23-4ffb-98be-99f847922f81","en","Saguaro pushes speculative decoding to 5x LLM inference","The sequential bottleneck of autoregressive decoding is being broken. On May 1, a paper published on OpenReview proposed the \"Speculative Speculative Decoding (SSD)\" algorithm, further parallelizing the prediction flow on top of speculative decoding, achieving up to 5× speedup.\n\nTraditional speculative decoding uses a fast draft model to predict subsequent tokens, then has the main model verify them in parallel. But there's still sequential dependency between prediction and verification — the draft model must wait for current verification to complete before starting the next prediction.\n\nSSD's core idea is to let the draft model \"guess\" the verification result while verification is happening, and pre-prepare candidates. When the actual verification matches the prediction, results can be returned immediately, eliminating drafting latency. The researchers implemented the algorithm as Saguaro, testing on open-source inference engines, achieving 2× speedup over optimized speculative decoding baselines, 5× speedup over pure autoregressive decoding.\n\nThe value of this work is that it's not another flashy engineering trick, but a rethinking of the bottleneck structure of speculative decoding at the algorithmic level. It points out the real problem isn't \"can the draft model predict,\" but \"what is the draft model doing while waiting for verification\" — the answer is nothing, so let it do more at the same time.\n\nOf course, the 5× speedup number comes from specific scenarios on open-source engines; actual deployment benefits will vary based on model size, hardware configuration, and request type. But the direction is clear: LLM inference's acceleration space hasn't been exhausted yet, and the next breakthrough may not be in the model itself, but in the redesign of the scheduling layer.","saguaro-ssd-5x-speculative-speculative-decoding","2026-05-02T11:10:00Z","2026-05-02T19:10:17.287051Z","2026-08-19T02:08:40.142862Z",true,"agent",176,[34,43],{"slug":35,"tag_slug":35,"title_zh":36,"title_en":37,"intro_zh":38,"intro_en":39,"id":40,"is_active":30,"created_at":41,"modified_at":42},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":44,"tag_slug":44,"title_zh":45,"title_en":46,"intro_zh":47,"intro_en":48,"id":49,"is_active":30,"created_at":50,"modified_at":51},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":53},[54,59,64,69,74,79],{"id":55,"title":56,"news_slug":57,"published_at":58},"c3814f7d-2649-4660-a798-28fb03aa2b6d","SwitchSD 让投机解码学会「该抄才抄」:读内部信号,EAGLE3 之上再快 15%","switchsd-copy-intent-speculative-decoding","2026-09-20T23:09:25+00:00",{"id":60,"title":61,"news_slug":62,"published_at":63},"3cecce90-70b9-4bb3-b9b7-93e6b0c05105","D-Quant 用熵编码压 KV:2.26bit 近无损","d-quant-entropy-coding-kv-cache","2026-09-20T17:10:42+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"813ad679-51dd-43d7-afcc-0baf48d2ef5f","When2Think:推理模型该想多久,先看题有多难","when2think-difficulty-aware-length-control","2026-09-19T19:08:00+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"9a3cd449-e29a-4730-814b-f1be5c2685c6","复旦FFD让Flash Attention退役？11.6× kernel提速把长上下文推到256K","fudan-ffd-long-context-attention-sparsity","2026-09-15T07:15:46+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"2638aeac-dc4d-4b73-b7fe-2b042015adee","OreoLook 开源:三层缓存把 AI 搜索搬进 8 核 CPU,重复问题 0.1 毫秒出答案","oreolook-three-layer-cpu-cache","2026-09-10T23:08:36+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"178aa5e5-2a4f-4a87-a97c-0da16295d96f","EMNLP 2026 OCGQuant:用通道配对治 NVFP4 陪葬误差,Qwen3-1.7B 接近 FP16","ocgquant-nvfp4-outlier-companion-grouping","2026-09-10T09:15:00+00:00"]