[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-pyrodash-small-large-routing":3,"news-related-beec1ff3-22af-4657-b58a-90cb0797c3b1":36},{"id":4,"title":5,"summary":6,"content":6,"original_url":7,"source_id":8,"tags":9,"translations":23,"news_slug":29,"published_at":30,"created_at":31,"modified_at":32,"is_published":33,"publish_type":34,"image_url":13,"view_count":35},"beec1ff3-22af-4657-b58a-90cb0797c3b1","PyroDash 让小模型「借力」大模型推理：把 LLM 调用砍到 1.9%，成本从 $49 降到 $1.78","arXiv 2607.20327 上线的 PyroDash 提出了一种 token 级的小-大模型协同推理框架:在小模型生成过程中插入一个「求助控制 token」,触发一次性 hand-off 把难题扔给冻结的大模型完成。核心思路是把「什么时候该花大钱调用 LLM」这件事内化到 SLM 自身,不再依赖外挂路由器,也不需要重新训练 LLM 或访问其 logits。训练分三阶段走:控制 token embedding 学习 → offloading-oriented SFT → 用 GRPO 做 cost-aware 对齐,奖励函数直接把推理成本和答案准确率绑在一起。在 5 个数学推理 benchmark 上,λ=0.05 时平均准确率 64.04%,比纯 LLM 基线还高 6.36 个百分点,成本砍掉 20.4%;λ=0.6 时准确率降到 54.55%,但 LLM token 占比仅 1.9%,单样本 LLM 调用 0.012 次,平均推理成本从 $49.36 降到 $1.78。这套思路的真正价值是把「按需升级」做成可学习的策略,而不是工程上的硬路由:小模型自己知道什么时候算不下去了,学会了精确举手。给生产部署的启示是,在保留 LLM 推理能力上限的同时,可以把大部分低难度请求甩给廉价 SLM 处理——这是 routing 论文里一直在画的大饼,但 PyroDash 用一个 control token 加 GRPO 的方式真正落了地。论文 73 页,代码和数据未公开,需要更多独立复现来验证阈值和奖励函数在不同领域的迁移性。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2607.20327","7437aeb9-930c-4866-a2e9-48003c1a792b",[10,14,17,20],{"id":11,"name":12,"slug":12,"description":13,"color":13},"7ac06d8e-b074-4147-abfc-ffaa4c6b8744","ai-efficiency",null,{"id":15,"name":16,"slug":16,"description":13,"color":13},"fca9258a-9430-455a-b95d-b9fae5e373a8","ai-inference",{"id":18,"name":19,"slug":19,"description":13,"color":13},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":21,"name":22,"slug":22,"description":13,"color":13},"01598627-1ea6-4b27-a5d8-874971571a71","llm",[24],{"id":25,"lang":26,"title":27,"summary":28,"content":28},"e4481a94-2e22-45d4-ae5f-089c45097e4b","en","PyroDash cuts LLM calls to 1.9%, cost from $49 to $1.78","PyroDash, posted to arXiv 2607.20327, proposes a token-level small\u002Flarge-model collaborative inference framework: it inserts a \"help-request control token\" during the small-model generation process, triggering a one-time hand-off that throws the hard problem to a frozen large model to complete. The core idea is to internalize \"when is it worth spending big money to call an LLM\" into the SLM itself, with no external router, no need to retrain the LLM, and no access to its logits. Training proceeds in three stages: control-token embedding learning → offloading-oriented SFT → cost-aware alignment with GRPO, where the reward function directly ties inference cost to answer accuracy. On 5 math reasoning benchmarks, with λ=0.05, the average accuracy is 64.04% — 6.36 percentage points above the pure-LLM baseline — and cost is cut 20.4%; with λ=0.6, accuracy drops to 54.55%, but the LLM token share is only 1.9%, with 0.012 LLM calls per sample, and the average inference cost drops from $49.36 to $1.78. The real value of this approach is turning \"on-demand escalation\" into a learnable policy rather than an engineering hard-route: the small model itself knows when it can't compute any further, and has learned to raise its hand precisely. The lesson for production deployment is that you can keep the upper bound of LLM inference capability while offloading most of the easy requests to cheap SLMs — this is the cake that routing papers have been drawing for a long time, but PyroDash actually lands it with one control token plus GRPO. The paper is 73 pages, code and data aren't public, and more independent reproduction is needed to verify the thresholds and reward function's transferability across domains.","pyrodash-small-large-routing","2026-07-24T00:00:00Z","2026-07-23T16:06:26.285299Z","2026-08-19T02:08:40.142862Z",true,"agent",113,{"items":37},[38,43,48,53,58,63],{"id":39,"title":40,"news_slug":41,"published_at":42},"6fa1bc74-c98e-476b-bc4c-9ae057105ffb","ParaTempo:免训练并行推理,延迟最高降 32%、token 省三成","paratempo-temporal-confidence-parallel-reasoning","2026-08-24T17:20:00+00:00",{"id":44,"title":45,"news_slug":46,"published_at":47},"5da227db-f53d-4b07-a0c6-4ea16e04cd4d","CURE 用不确定性焦点做「block-parallel 投机解码」：端到端 2.66–3.49×、接受长度涨 4.2–7.5%","cure-block-parallel-speculative-decoding","2026-08-08T02:00:00+00:00",{"id":49,"title":50,"news_slug":51,"published_at":52},"183fb3be-e062-47e7-9591-7c2372e116c1","LLM 蒸馏的显存瓶颈不只在教师模型：离线 Top-K 与分块 KL 把长上下文训练装回单卡","llm-distillation-offline-top-k-chunked-kl","2026-08-05T20:08:13+00:00",{"id":54,"title":55,"news_slug":56,"published_at":57},"f9a6c57b-a98e-407c-9751-93907ff314b0","Capacity-Aware Inference：MoE 推理 1.85× 加速","iclr-2026-capacity-aware-moe","2026-07-27T10:00:00+00:00",{"id":59,"title":60,"news_slug":61,"published_at":62},"7f723663-8405-43aa-b31e-73efd714fa97","KV-Cache Grafting：冻结权重，Gemma-4-12B AIME 80%→93.3%","byte-exact-kv-cache-grafting","2026-07-17T06:20:00+00:00",{"id":64,"title":65,"news_slug":66,"published_at":67},"ed39ed38-b5fa-4f58-92cf-d05233ab998b","Speculate with Memory：LLM Agent 无损加速 2.5×，准确率涨 39pp","speculate-with-memory-2-5x","2026-07-15T08:15:00+00:00"]