[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-lfm2-5-dspark-draft-models":3,"news-related-92433e6b-113a-4ada-af77-fbb8995a9850":38},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"92433e6b-113a-4ada-af77-fbb8995a9850","LFM2.5-DSpark 开源:300M 草稿模型让端侧推理快 2.87 倍,输出零损耗","Liquid AI 为 LFM2.5 三款模型开源 DSpark 投机解码草稿模型:约 300M 参数,GPU 吞吐最高 3.18 倍、MacBook 端侧 2.87 倍,函数调用延迟平均降 57%,输出与原模型逐 token 一致,llama.cpp\u002FSGLang 当天即用。","8 月 20 日,Liquid AI 在 Hugging Face 上发布了 LFM2.5 家族的 DSpark 草稿模型 checkpoints,覆盖三款目标模型:LFM2.5-1.2B-Instruct、LFM2.5-2.6B 和 LFM2.5-8B-A1B。这不是新基座,而是给现有模型外挂投机解码引擎——用一个约 300M 参数的 draft 模型先生成候选 token,再由目标模型一次前向批量验证,以极小内存代价换大幅提速。\n\n## 官方数字:GPU 3.18 倍,端侧 2.87 倍\n\n按 Liquid AI 的测试(SGLang,单张 H100 80GB,BF16),吞吐最高提升 3.18 倍;端侧用 llama.cpp + Metal 在 M4 Max MacBook Pro 上跑 FP16 GGUF,最高 2.87 倍。以 LFM2.5-2.6B 为例,MacBook 上 MATH500 从 61 提到 137 tok\u002Fs,MT-Bench 从 62 到 123 tok\u002Fs,官方称约 140 tok\u002Fs 已超过多数专有云模型的交互吞吐。更实际的是:多工具调用场景下,LFM2.5-2.6B 的函数调用延迟平均降 57%。\n\n## DSpark 的三件套\n\nLLM 解码的瓶颈通常不是算力,而是权重从 DRAM 进 SRAM 的带宽。投机解码让轻量 draft 模型连续产候选、目标模型一次验证多个,把权重加载成本摊到多个 token 上。DSpark 把这件事拆成三个组件:DFlash 风格的并行 backbone,以目标模型的上下文特征为条件,一次前向产出所有 draft token 的隐状态;一个建模相邻 token 间马尔可夫链的轻量顺序头,补上 token 间依赖、提高后位接受率;以及一个置信度调度的验证器,预测每个 token 的存活概率,验证不划算时提前剪枝。工程细节有点反直觉:训练跑了 15 个 epoch,最终选的不是 loss 最低、而是接受率最高的点——draft 模型的 KPI 是\"被接受\",不是自己算得准。\n\n## 输出零损耗是\"构造性\"的\n\n很多人担心投机解码牺牲质量,这里不用猜:贪心解码下,draft token 只有与目标模型分布一致才会被接受,被拒位置由目标模型自己的 token 补位,输出序列按构造就与基线完全一致,benchmark 精度(pass@1、exact match)不变。这比\"几乎不掉点\"的量化路线承诺更硬。\n\n## MoE 在端侧撞了墙\n\n有意思的是 8B-A1B(MoE 版):它的接受率反而是三款里最高的(MT-Bench 8.52\u002F10,MATH500 8.27\u002F10),H100 上也打出了 3.18 倍的最高加速,但 M4 Max 端侧平均只提升 18%。官方解释是 llama.cpp Metal 后端的 MoE 实现限制——一次验证 k 个 token 会激活更多专家、带来更多权重流量。换句话说,dense 模型在端侧吃满红利,MoE 还得等推理框架先补课。\n\nllama.cpp(PR #27383)和 SGLang(PR #31041)的集成都已开源上游,checkpoints 提供 Safetensors 和 GGUF 两种格式。与其追下一个更大的基座,不如把\"验证是免费的\"吃透——对端侧 Agent 来说,延迟降一半比榜单涨一分值钱。详见[Liquid AI 官方博客](https:\u002F\u002Fhuggingface.co\u002Fblog\u002FLiquidAI\u002Flfm25-dspark)。\n","https:\u002F\u002Fhuggingface.co\u002Fblog\u002FLiquidAI\u002Flfm25-dspark","511bb1e6-a31f-4dc1-929b-9a7582e67447",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":19,"name":20,"slug":20,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":22,"name":23,"slug":23,"description":14,"color":14},"b9bd9039-fcdb-41a8-b85b-fc1587def2b9","open-source",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"0272f457-a317-4519-a88c-0123be429dfb","en","LFM2.5-DSpark Open-Sourced: 300M Draft Models Speed Up On-Device Inference 2.87x With Zero Output Loss","Liquid AI open-sources DSpark speculative decoding draft models for three LFM2.5 models: ~300M parameters, up to 3.18x GPU throughput and 2.87x on MacBook, 57% lower function-calling latency on average, token-identical output, day-one llama.cpp\u002FSGLang support.","On August 20, Liquid AI released DSpark draft model checkpoints for the LFM2.5 family on Hugging Face, covering three target models: LFM2.5-1.2B-Instruct, LFM2.5-2.6B, and LFM2.5-8B-A1B. This is not a new base model — it bolts a speculative decoding engine onto existing ones. A roughly 300M-parameter draft model generates candidate tokens first, and the target model verifies them in a single forward pass, trading minimal memory for a large decoding speedup.\n\n## The official numbers: 3.18x on GPU, 2.87x on-device\n\nIn Liquid AI's tests (SGLang, single H100 80GB, BF16), throughput improved by up to 3.18x; on-device with llama.cpp + Metal on an M4 Max MacBook Pro running FP16 GGUF, up to 2.87x. Take LFM2.5-2.6B: on the MacBook, MATH500 goes from 61 to 137 tok\u002Fs and MT-Bench from 62 to 123 tok\u002Fs — the company says the ~140 tok\u002Fs level already exceeds the interactive throughput of most proprietary cloud models. More practically for agent developers: across multi-tool scenarios, function-calling latency for LFM2.5-2.6B drops by 57% on average.\n\n## The three pieces of DSpark\n\nThe bottleneck in LLM decoding is usually not compute but the bandwidth of streaming weights from DRAM into SRAM. Speculative decoding lets a lightweight draft model produce candidates continuously while the target verifies several at once, amortizing weight-loading cost across multiple tokens. DSpark splits this into three components: a DFlash-style parallel backbone conditioned on the target model's context features, producing hidden states for all draft tokens in one forward pass; a lightweight sequential head modeling a Markov chain between neighboring tokens, adding inter-token dependency and raising acceptance at later positions; and a confidence-scheduled verifier that predicts each token's survival probability and prunes low-confidence suffixes when verification would cost more than it saves. One engineering detail is counterintuitive: training ran 15 epochs, and the final pick was not the lowest-loss checkpoint but the highest-acceptance one — a draft model's KPI is being accepted, not being accurate on its own.\n\n## Zero quality loss is by construction\n\nMany worry that speculative decoding sacrifices quality. No guessing needed here: under greedy decoding, a draft token is accepted only if it matches the target model's distribution, and rejected positions are filled by the target's own tokens. The emitted sequence is therefore identical to baseline greedy by construction, leaving benchmark accuracy (pass@1, exact match) unchanged. That is a stronger promise than the \"almost no quality drop\" framing of quantization approaches.\n\n## MoE hits a wall on-device\n\nInterestingly, the 8B-A1B (MoE version) has the highest acceptance of the three (MT-Bench 8.52\u002F10, MATH500 8.27\u002F10) and delivered the best 3.18x speedup on H100 — but only an 18% average improvement on the M4 Max. The official explanation: limitations of llama.cpp's Metal backend for MoE — verifying k tokens activates more experts and thus more weight traffic. In other words, dense models fully harvest the speculative decoding dividend on-device, while MoE has to wait for inference frameworks to catch up.\n\nBoth integrations — llama.cpp (PR #27383) and SGLang (PR #31041) — are open-sourced upstream, and checkpoints ship in Safetensors and GGUF formats. Rather than chasing the next bigger base model, it pays to internalize that \"verification is free\" — for on-device agents, halving latency is worth more than a point on a leaderboard. See the [Liquid AI blog post](https:\u002F\u002Fhuggingface.co\u002Fblog\u002FLiquidAI\u002Flfm25-dspark).\n","lfm2-5-dspark-draft-models","2026-08-21T21:10:00Z","2026-08-21T21:08:27.339732Z","2026-08-21T21:08:27.339741Z",true,"agent",68,{"items":39},[40,45,50,55,60,65],{"id":41,"title":42,"news_slug":43,"published_at":44},"cac485ab-a429-4ebc-88c6-a1f924f978ff","AWS 开源 KeysAndValues:微调时就让模型学会“遗忘”,单张 A100 撑住 128K","aws-keysvalues-sparse-attention-finetuning","2026-08-26T05:20:00+00:00",{"id":46,"title":47,"news_slug":48,"published_at":49},"c94bdf86-5de9-49fe-8c98-0f5c47611bfe","SGLang v0.5.18 发布:大模型冷启动提速 2.38 倍,710 个 PR 都改了什么","sglang-v0-5-18-cold-start-2-38x","2026-08-24T23:15:00+00:00",{"id":51,"title":52,"news_slug":53,"published_at":54},"926f89fc-5ed5-4170-bfac-931d3a31b6a4","腾讯混元开源 AngelSpec 投机解码框架：DFly 在 Hy3-A21B 上取得 1.98–2.40× 加速","tencent-angelspec-spec-decoding-hy3-dfly","2026-07-30T00:00:00+00:00",{"id":56,"title":57,"news_slug":58,"published_at":59},"894787b7-8799-4c22-8e20-5e04db8f7181","DeepSeek V4 GA 把分时电价塞进大模型 API:CSA + HCA 撑起 1M 上下文,峰值翻倍只是商业答案","deepseek-v4-ga-time-of-use-pricing","2026-07-24T02:00:00+00:00",{"id":61,"title":62,"news_slug":63,"published_at":64},"518d0be2-1220-4920-98fd-dee9df27a43d","VIDRAFT VKUE 把同一份 34.7B 稀疏 MoE 权重从 B200 跑到裸 CPU","vidraft-vkue-cpu-moe","2026-07-12T10:00:00+00:00",{"id":66,"title":67,"news_slug":68,"published_at":64},"f8a33525-3846-46e2-9f88-05be407dbe2f","Liquid AI 用 Antidoom 把推理模型的 doom loop 压到 1%：一次训练只改 1 个 token","liquid-ai-antidoom"]