[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-llm-inference-or-paradigm-heuristics":3,"topics-all":36,"news-related-2e1d1723-4cea-4621-965e-9514d08a9013":55},{"id":4,"title":5,"summary":6,"content":6,"original_url":7,"source_id":8,"tags":9,"translations":23,"news_slug":29,"published_at":30,"created_at":31,"modified_at":32,"is_published":33,"publish_type":34,"image_url":13,"view_count":35},"2e1d1723-4cea-4621-965e-9514d08a9013","LLM推理服务正在淘汰「启发式」：运筹学视角下的新优化范式","LLM推理服务规模已达日均数十亿次请求，但核心算法仍沿用通用分布式系统的启发式策略——路由用最短队列、调度用FIFO、缓存用LRU。arXiv一篇5月2日发表的论文指出，LLM推理的结构独特性（动态增长的KV Cache、prefill-decode相位不对称、未知输出长度、连续批处理）使得通用启发式错失了大量优化空间。该论文认为，运筹学与ML系统的交叉领域已证明，原则性方法可以在提供理论保证的同时匹配或超越启发式性能。\n\n具体来说，当前LLM serving系统（如vLLM和SGLang）的核心算法几乎没变：请求路由还是最短队列或轮询，调度默认FIFO，KV Cache回收用LRU。这些通用策略完全忽视了LLM推理的特殊结构。论文主张，必须为LLM serving开发能够捕捉这些特征的数学模型，设计具有可证明性能保证的算法，而非在某些场景有效但会在其他场景不可预测地失败的启发式方法。\n\n在MoE负载均衡场景中这个问题尤为突出：当token集中在少数热门专家时，托管这些专家的GPU成为瓶颈，其他GPU只能空闲等待。当前主要的平衡策略是辅助损失函数，惩罚token跨专家的不均匀分布，但这会引入与主语言建模目标冲突的梯度干扰。论文认为，需要更原则性的方法来处理这类问题。\n\n从工业角度看，这篇论文的价值在于它不是空谈理论——它明确指出了LLM serving中存在的具体决策问题（请求路由、调度、缓存管理、负载均衡、容量规划、资源分配），这些问题都适合形式化分析。当vLLM和SGLang等推理引擎架构趋于稳定，算法层面的创新将成为持久的投资，不会因为系统增量更新而需要重新设计。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2605.01280","7437aeb9-930c-4866-a2e9-48003c1a792b",[10,14,17,20],{"id":11,"name":12,"slug":12,"description":13,"color":13},"40269b40-7942-4650-9672-ed2e6524d37a","ai-technology",null,{"id":15,"name":16,"slug":16,"description":13,"color":13},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",{"id":18,"name":19,"slug":19,"description":13,"color":13},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":21,"name":22,"slug":22,"description":13,"color":13},"01598627-1ea6-4b27-a5d8-874971571a71","llm",[24],{"id":25,"lang":26,"title":27,"summary":28,"content":13},"716c4fe6-cfbc-4e02-b999-8794d1b05dec","en","LLM serving retires heuristics: the operations-research lens","LLM inference services have reached tens of billions of requests per day, but core algorithms still rely on the heuristic strategies of general-purpose distributed systems — shortest-queue routing, FIFO scheduling, LRU caching. An arXiv paper published on May 2 points out that LLM inference's structural uniqueness (dynamically growing KV Cache, prefill-decode phase asymmetry, unknown output length, continuous batching) means general heuristics miss substantial optimization space. The paper argues that the intersection of operations research and ML systems has proven that principled methods can match or exceed heuristic performance while providing theoretical guarantees.\n\nSpecifically, the core algorithms of current LLM serving systems (like vLLM and SGLang) have barely changed: request routing is still shortest-queue or round-robin, scheduling defaults to FIFO, KV cache reclamation uses LRU. These general strategies completely ignore LLM inference's special structure. The paper advocates that mathematical models capable of capturing these features must be developed for LLM serving, with algorithms that have provable performance guarantees, rather than heuristics that may work in some scenarios but fail unpredictably in others.\n\nThis problem is especially acute in MoE load-balancing scenarios: when tokens concentrate on a few hot experts, the GPUs hosting those experts become the bottleneck, while other GPUs can only sit idle. Current main balancing strategies use auxiliary loss functions to penalize uneven token distribution across experts, but this introduces gradient interference conflicting with the main language-modeling objective. The paper argues more principled methods are needed to handle these issues.\n\nFrom an industry perspective, the paper's value is that it's not empty theoretical talk — it explicitly identifies specific decision problems in LLM serving (request routing, scheduling, cache management, load balancing, capacity planning, resource allocation), all of which are amenable to formal analysis. As inference engine architectures like vLLM and SGLang stabilize, algorithm-level innovation becomes a durable investment, not requiring redesign with every system incremental update.","llm-inference-or-paradigm-heuristics","2026-05-16T08:25:00Z","2026-05-16T16:25:33.376054Z","2026-08-19T02:08:40.142862Z",true,"agent",231,[37,46],{"slug":38,"tag_slug":38,"title_zh":39,"title_en":40,"intro_zh":41,"intro_en":42,"id":43,"is_active":33,"created_at":44,"modified_at":45},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":47,"tag_slug":47,"title_zh":48,"title_en":49,"intro_zh":50,"intro_en":51,"id":52,"is_active":33,"created_at":53,"modified_at":54},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":56},[57,62,67,72,77,82],{"id":58,"title":59,"news_slug":60,"published_at":61},"217f417d-1b9c-475b-99f4-e21e7c909711","MHAR 把 Transformer 残差流从「单车道」拆成 H 条独立路由:子空间第一次有权自己挑历史层","multi-head-attention-residuals-mhar","2026-08-01T07:30:00+00:00",{"id":63,"title":64,"news_slug":65,"published_at":66},"623f7e16-ef9a-43fc-9303-d01bfd60d8fe","把 LLM 推理拆成四层架构：62 页综述给「Token 运营」补一条产业视角","token-operations-four-layer-62-page-survey","2026-06-18T14:33:00+00:00",{"id":68,"title":69,"news_slug":70,"published_at":71},"4e43e35d-a808-4125-be31-69cadedc61f1","PoLar 把 LLM 层变成可调积木：动态跳层+复读，3B 模型数学推理涨 60+ 个百分点","polar-icml-2026-3b-math-62pp-jump","2026-06-15T14:00:00+00:00",{"id":73,"title":74,"news_slug":75,"published_at":76},"fc93d022-8522-4396-a047-c9ba8fc1821c","VIA-SD 入选 ICML 2026：投机解码终于有了「瘦验证器」，推理再快 20%","via-sd-icml-2026-slim-verifier-20pct","2026-06-11T20:15:00+00:00",{"id":78,"title":79,"news_slug":80,"published_at":81},"3a9a8c69-d668-4d2c-ae82-caeba45aa2d5","MIT新方法利用计算空闲周期：推理模型训练速度翻倍，能耗减半","mit-rllm-idle-cycle-2x-train-half-energy","2026-05-22T08:10:00+00:00",{"id":83,"title":84,"news_slug":85,"published_at":86},"60859a35-6e56-432b-82cc-7edc146200ef","LLM推理评估新范式：当「能源墙」取代「算力墙」","llm-inference-energy-wall-token-production","2026-05-14T07:01:00+00:00"]