[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-paratempo-temporal-confidence-parallel-reasoning":3,"news-related-6fa1bc74-c98e-476b-bc4c-9ae057105ffb":38},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"6fa1bc74-c98e-476b-bc4c-9ae057105ffb","ParaTempo:免训练并行推理,延迟最高降 32%、token 省三成","上海交大团队在 arXiv 发布免训练异步并行推理框架 ParaTempo,用\"时间置信度\"信号动态剪枝、提前退役、再分配并行推理分支,数学与科学推理基准上平均延迟降 21.8-32.2%、token 用量降 18.1-30.3%,精度保持竞争力,代码已开源。","并行推理正在成为推理模型的标配:同一道题开多条分支各自解,再投票选答案,准确率确实上去了。但代价也是实打实的——开几条分支,算力就翻几倍,而且大多数方法里所有分支都要跑到底才算数。上海交大团队 8 月 17 日挂在 arXiv 上的 ParaTempo,给这个问题提供了一个相当干净的解法:不训练、不改模型,只靠一个叫「时间置信度」的信号,把并行推理的开销砍下来一大截——平均延迟降 21.8-32.2%,总 token 用量降 18.1-30.3%,精度保持竞争力。\n\n## 老问题:控制信号要么太迟,要么太噪\n\n先看他们攻击的痛点。管理并行分支的已有方法,大多依赖三类信号:最终答案共识(要等所有分支跑完才知道谁对)、局部 token 置信(单个 token 的概率,和推理真实进展关联弱)、孤立的中间探测(噪声太大,撑不起分支级的动态控制)。这三类信号共同的毛病,论文的概括是:延迟、与推理进展脱节、噪声大。而分支级的调度——什么时候砍掉一条分支、什么时候提前收工——需要的是一个既便宜又能实时跟上推理进度的信号。\n\n## 时间置信度:一个信号驱动四个动作\n\nParaTempo 的核心是 temporal confidence(时间置信度),一个分支局部的「答案空间收敛度」度量。做法是:周期性地让每条分支给出一个试探性的答案概率分布,然后量化最近的中间探测有多集中在同一个主导答案上。一条分支反反复复都指向同一个答案,说明它已经收敛;答案还在漂,说明它还没想清楚。\n\n有了这个信号,ParaTempo 的整个控制流程都由它驱动,共四个动作:置信度低的分支直接剪枝;持续锁定同一主导答案的分支提前退役,不再烧 token;腾出来的算力通过 fork 新分支再分配;一旦置信度加权的投票集中,全局停止生成。全程不需要分支之间同步——这是个异步框架,每条分支按自己的节奏收敛。\n\n值得强调的是 training-free 这一点:不动模型权重、不需要额外训练,理论上可以直接套在现成的大推理模型外面当调度器用。论文还顺带验证了信号本身的质量:相比 token 级和瞬时信号,时间置信度的时间稳定性更强,对未来分支收敛的预测力也更好。\n\n## 效果与边界\n\n在数学和科学推理基准上,ParaTempo 把平均延迟降 21.8-32.2%、总 token 用量降 18.1-30.3%,同时保持有竞争力的准确率。注意论文的措辞是 \"maintaining competitive accuracy\"——是「有竞争力」,不是「零损失」;对生产环境来说,用三成延迟换一个可接受的精度取舍,这笔账通常算得过来。\n\n边界也直说:这套机制的前提是「答案分布会收敛」。对答案空间本来就发散、或需要长链条多步验证才能收敛的任务,时间置信度这个判据会变弱。另外它是推理时的调度框架,不解决模型本身的能力上限。\n\n代码和数据集已开源(github.com\u002FScottZhang812\u002FParaTempo),论文见 [arXiv:2608.16425](https:\u002F\u002Farxiv.org\u002Fabs\u002F2608.16425)。\n\n## 所以呢\n\n推理成本战打到今天,注意力大多集中在模型侧:蒸馏、量化、MoE。ParaTempo 代表另一条路——推理调度侧的免费午餐。一个外挂的、免训练的分支调度器,不动模型就能省三成开销;对任何已经在跑并行推理(best-of-N、自一致性投票)的服务来说,这类调度层优化可能是 ROI 最高的一档。「推理效率」的视野,值得从模型内部挪一点到推理编排层。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2608.16425","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"7ac06d8e-b074-4147-abfc-ffaa4c6b8744","ai-efficiency",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"fca9258a-9430-455a-b95d-b9fae5e373a8","ai-inference",{"id":19,"name":20,"slug":20,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":22,"name":23,"slug":23,"description":14,"color":14},"b9bd9039-fcdb-41a8-b85b-fc1587def2b9","open-source",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"f9a180aa-d1fe-4299-bb73-6e73cd69b9cc","en","ParaTempo: Training-Free Parallel Reasoning Cuts Latency by 32%","SJTU's ParaTempo steers training-free parallel reasoning with one signal: latency down 21.8-32.2%, tokens down 18.1-30.3%, competitive accuracy.","Parallel reasoning has become standard equipment for reasoning models: spawn multiple branches on the same problem, solve independently, then vote. Accuracy goes up, but so does the bill — every extra branch multiplies compute, and in most methods all branches must run to completion. ParaTempo, posted to arXiv on Aug 17 by a Shanghai Jiao Tong University team, offers a clean fix: no training, no model changes, just one signal called \"temporal confidence\" — cutting average latency by 21.8-32.2% and total token usage by 18.1-30.3% while maintaining competitive accuracy.\n\n## The old problem: control signals are late or noisy\n\nThe paper targets a well-known pain point. Existing approaches to managing parallel branches rely on three kinds of signals: final-answer consensus (you only learn which branch was right after everything finishes), local token confidence (weakly tied to actual reasoning progress), and isolated intermediate probes (too noisy for dynamic, branch-level control). All three share the same flaws, in the authors' words: delayed, weakly tied to reasoning progress, or noisy. Branch-level scheduling — when to kill a branch, when to stop early — needs a signal that is both cheap and tracks reasoning progress in real time.\n\n## Temporal confidence: one signal, four actions\n\nThe core idea is temporal confidence, a branch-local measure of answer-space convergence. Each branch is periodically probed for a tentative answer probability distribution; temporal confidence quantifies how sharply the recent intermediate probes concentrate on one dominant answer. A branch that keeps pointing at the same answer has converged; one still drifting has not.\n\nFrom this single signal, ParaTempo drives its entire control process through four actions: low-confidence branches are pruned; branches that persistently commit to a dominant answer are retired early, burning no more tokens; freed computation is reallocated by forking new branches; and once the confidence-weighted vote concentrates, generation stops globally. No synchronization among reasoning trajectories is required — this is an asynchronous framework where each branch converges at its own pace.\n\nThe training-free property matters: no weight updates, no auxiliary training, so it can in principle be wrapped around an existing large reasoning model as an external scheduler. The paper also validates the signal itself: temporal confidence shows stronger temporal stability and better predictive power for future branch convergence than token-level and instantaneous signals.\n\n## Results and limits\n\nOn challenging mathematical and scientific reasoning benchmarks, ParaTempo reduces average latency by 21.8-32.2% and total token usage by 18.1-30.3% while maintaining competitive accuracy. Note the wording: \"competitive,\" not \"zero loss\" — for production, trading roughly a third of latency for an acceptable accuracy trade-off usually pays off.\n\nThe limits are real: the mechanism assumes answer distributions converge. For tasks with inherently divergent answer spaces, or those needing long multi-step verification before converging, the criterion weakens. And it is an inference-time scheduler — it does not raise the model's underlying capability ceiling.\n\nCode and dataset are open-sourced at github.com\u002FScottZhang812\u002FParaTempo; the paper is at [arXiv:2608.16425](https:\u002F\u002Farxiv.org\u002Fabs\u002F2608.16425).\n\n## So what\n\nThe inference-cost war has mostly focused on the model side — distillation, quantization, MoE. ParaTempo points at another path: free wins in the scheduling layer. An external, training-free branch scheduler that saves roughly 30% without touching the model is, for any service already running parallel inference (best-of-N, self-consistency voting), possibly the highest-ROI optimization available. It is worth shifting some attention from inside the model to the inference orchestration layer.","paratempo-temporal-confidence-parallel-reasoning","2026-08-24T17:20:00Z","2026-08-24T17:10:23.202558Z","2026-08-24T17:10:23.202572Z",true,"agent",42,{"items":39},[40,45,50,55,60,65],{"id":41,"title":42,"news_slug":43,"published_at":44},"f9a6c57b-a98e-407c-9751-93907ff314b0","Capacity-Aware Inference：MoE 推理 1.85× 加速","iclr-2026-capacity-aware-moe","2026-07-27T10:00:00+00:00",{"id":46,"title":47,"news_slug":48,"published_at":49},"68072ee1-fc37-4064-ab18-09550ae72d1b","GLM-5.3-Flash 把 320B MoE 跑在国产芯片上:Flash 价位和 $0.15 API 的混合注意力栈","glm-5-3-flash-chinese-chips-hybrid-attention","2026-08-27T03:00:00+00:00",{"id":51,"title":52,"news_slug":53,"published_at":54},"4aa9534a-778e-4cd7-8194-fdf3097249b8","OpenAI Jalapeño Hot Chips 实测:峰值每瓦 1.9×,延迟压到 1 秒","openai-jalapeno-hot-chips-benchmark-2026","2026-08-26T02:00:00+00:00",{"id":56,"title":57,"news_slug":58,"published_at":59},"491f4904-c854-4925-b3e3-e34b8afd5e50","KDA+MLA 混合栈下沉到 1.3B 激活:Ling-3.0-tiny 把 MoE 端侧化,INT4 跑出 115 tok\u002Fs","ling-3-tiny-kda-mla-edge-deployment","2026-08-18T00:00:00+00:00",{"id":61,"title":62,"news_slug":63,"published_at":64},"259d91b2-ed6b-4af8-8f2a-f759b84cc617","蚂蚁 Ling-3.0 Flash：124B\u002F5.1B MoE 的 Agent 生产级模型","inclusionai-ling-3-flash-hybrid-linear-moe-agent","2026-08-14T08:00:00+00:00",{"id":66,"title":67,"news_slug":68,"published_at":69},"5da227db-f53d-4b07-a0c6-4ea16e04cd4d","CURE 用不确定性焦点做「block-parallel 投机解码」：端到端 2.66–3.49×、接受长度涨 4.2–7.5%","cure-block-parallel-speculative-decoding","2026-08-08T02:00:00+00:00"]