[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-deepseek-deepinfer-open-source-inference-engine":3,"topics-all":38,"news-related-e880f146-5f06-4b07-a4da-e6a7391ddcff":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"e880f146-5f06-4b07-a4da-e6a7391ddcff","DeepSeek 开源 DeepInfer:生产推理引擎放出来,服务那一半还留在内部","DeepSeek 10 月 20 日以 Apache 2.0 协议开源 DeepInfer,即支撑其线上服务的高性能推理框架,目前只支持 V4.1 Flash,硬件限定英伟达 SM100 与华为昇腾 950。引擎部分完整放出,KV cache 缓存、Prefill\u002FDecode 分离与负载均衡仍留在内部。","很多公司的推理框架是论文的延伸:论文里跑通,服务里还要再调半年。DeepSeek 这次反着来——10 月 20 日以 Apache 2.0 协议开源 DeepInfer,即支撑其线上服务的高性能推理框架。仓库两天就显示 1.8 万星和 1900 个 fork,代码本身却只有 4 次提交。\n\n## 开源了什么,没开源什么\n\nREADME 写得很直白:这是「支撑 DeepSeek 大规模生产服务的高性能推理框架」,这次放出的是「离线推理,以及规模化服务所需的核心功能」。默认加载 DeepSeek-V4.1-Flash-1017,支持模型列表目前也只有这一个。\n\n能力清单不短:注意力数据并行与均衡上下文并行、MoE 专家并行加 EPLB、DSpark 推测解码与置信度调度验证、SWA 有界重放、异步调度(零开销批调度器)、批不变推理、确定性采样、结构化输出。算子层直接调用自家仓库 FlashMLA、DeepGEMM、DeepEP、TileKernels、DeepSelect,外加自写的 TileLang kernel。\n\n没放出来的同样标了出来:KV cache 缓存系统、Prefill\u002FDecode 分离、两侧数据并行副本间的负载均衡,理由都是与内部基础设施耦合紧密。拿到的是引擎,不是服务。同步开源的 deepseek-recipe 负责把请求转成 token 序列,用的是官方 API 同样的逻辑。\n\n## 硬件清单很短\n\n只有英伟达 SM100 和华为昇腾 950 NPU。SiliconANGLE 特别点出,Hopper 及更早的英伟达架构不在其中——手上只有 A100 或 H100 的团队暂时用不上。批不变推理和确定性采样目前也仅限 NVIDIA。\n\n## 两个设计上值得看的地方\n\n**SWA 有界重放。** V4.1 Flash 部分层采用滑动窗口注意力,这部分 KV Cache 只驻留内存。缓存未命中时,旧做法按「层数 × 窗口长度」重算;新做法只把每条 prompt 的最后 128 个 token 重跑一遍,注意力也只在这 128 个内计算,于是重算成本被固定,与上下文长度和层数解耦。按 SiliconANGLE 的说法,这让 prefill 总计算量接近减半。\n\n**DSpark 的置信度调度。** 轻量草稿模块先连续猜 5 个 token,目标模型批量验证,猜错的由大模型纠正。关键在于置信度头会预估每个草稿被接受的概率,低可能性的不再验证,把算力让给其他请求。同样据 SiliconANGLE,这让每步需检查的草稿 token 少了约 30%,DeepSeek 估算在自家生产负载上吞吐提升 4% 到 7%。\n\n## 数字要打折看\n\n据随代码发布的部署 Q&A,一个集群现在每天服务 155 万亿 token,按谷段定价折算约每 GPU 每小时 38 美元。这是 DeepSeek 自己的数据,所有公开性能跑分也都在英伟达硬件上完成:prefill 4 张 SM100、decode 64 张。同一篇报道提到,DeepSeek 描述的最小高效部署是 prefill 64 卡加 decode 64 卡。昇腾支持则跟它 9 月 30 日把 TileLang 带到华为芯片那件事连着,仓库里备了 requirements-ascend.txt 并要求 CANN 9.2。\n\n## 所以呢\n\n对多数团队,这份代码眼下不是拿来就上线的:一个模型、两张卡型,服务化那一半还在 DeepSeek 内部。但价值在另一头——它把头部厂商在生产里反复调出来的工程决策公开了。滑动窗口重放、置信度调度、前缀缓存,过去只能在闭源集群里猜,现在能直接读源码。\n\n还有一件事值得对照:这份代码放出前一天,美国国土安全部下令联邦文职机构移除 DeepSeek 产品,包括基于其技术的第三方服务。开源与地缘政治落在同一个 48 小时窗口,大概是理解这次发布的另一把钥匙。\n\n**素材来源**\n\n- DeepInfer 官方仓库(一手):https:\u002F\u002Fgithub.com\u002Fdeepseek-ai\u002FDeepInfer\n- SiliconANGLE 报道(独立二手):https:\u002F\u002Fsiliconangle.com\u002F2026\u002F10\u002F20\u002Fdeepseek-open-sources-deepinfer-the-inference-framework-behind-its-production-services\u002F","https:\u002F\u002Fgithub.com\u002Fdeepseek-ai\u002FDeepInfer","998df6db-96e6-4b8e-8be1-cfa00a6cd177",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"b52db7e9-7c58-42c3-9536-5132cb2f8f72","deepseek",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",{"id":19,"name":20,"slug":20,"description":14,"color":14},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":22,"name":23,"slug":23,"description":14,"color":14},"b9bd9039-fcdb-41a8-b85b-fc1587def2b9","open-source",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"ca0e3ccf-c18e-43a3-be61-6719a997bb02","en","DeepSeek open-sources DeepInfer: engine yes, serving stack no","DeepSeek released DeepInfer under Apache 2.0 on October 20, the engine behind its production services, limited to V4.1 Flash on SM100 and Ascend 950.","Most inference frameworks extend a paper: the numbers work in the paper, then the serving stack needs half a year of tuning. DeepSeek ran it the other way round. On October 20 it open-sourced DeepInfer under the Apache 2.0 license, the engine behind its production services. The repository showed 18,000+ stars and 1,900 forks within two days, while the code itself is four commits deep.\n\n## What shipped, and what did not\n\nThe README is blunt: DeepInfer is \"the high-performance inference framework that powers DeepSeek's large-scale production services\", and what ships now is \"offline inference with the core features needed for serving at scale\". The default checkpoint is DeepSeek-V4.1-Flash-1017, and it is the only supported model.\n\nThe feature list runs long: attention data parallelism and balanced context parallelism, MoE expert parallelism with EPLB, DSpark speculative decoding with confidence-scheduled verification, SWA bounded replay, asynchronous scheduling via a zero-overhead batch scheduler, batch-invariant inference, deterministic sampling, and structured outputs. At the kernel layer it calls DeepSeek's own FlashMLA, DeepGEMM, DeepEP, TileKernels and DeepSelect repos, plus custom TileLang kernels.\n\nWhat is left out is stated just as plainly: the KV cache system, prefill\u002Fdecode disaggregation, and load balancing across data-parallel replicas, with the reason given as tight coupling to internal infrastructure. What you get is the engine, not the service. A companion library, deepseek-recipe, converts requests into token sequences using the same logic as DeepSeek's official API.\n\n## A very short hardware list\n\nSupported hardware is NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs, nothing else. SiliconANGLE notes that Hopper and older NVIDIA architectures are absent, so teams holding only A100s or H100s cannot use this. Batch-invariant inference and deterministic sampling are also NVIDIA-only for now.\n\n## Two design choices worth reading\n\n**SWA bounded replay.** Parts of V4.1 Flash use sliding-window attention, and that slice of the KV cache stays in memory. On a cache miss, the old approach recomputed along \"number of layers times window length\". The new one replays only the final 128 tokens of each prompt, with attention computed inside those 128, pinning the recompute cost so it no longer scales with context length or depth. Citing DeepSeek, SiliconANGLE reports this nearly halves total prefill computation.\n\n**Confidence-scheduled verification in DSpark.** A lightweight draft module proposes five tokens at a time and the target model verifies them in a batch, correcting the first wrong one. A confidence head predicts how likely each draft is to be accepted, so low-probability drafts are skipped and the compute goes elsewhere. SiliconANGLE reports this cuts draft tokens checked per step by about 30%, with DeepSeek estimating a 4% to 7% throughput gain at its production load.\n\n## Discount the numbers\n\nPer a deployment Q&A published with the code, one cluster now serves 155 trillion tokens a day, about 38 USD per GPU per hour at off-peak pricing. Those are DeepSeek's own numbers, and every published benchmark ran on NVIDIA hardware: prefill on 4 SM100 GPUs, decode on 64. The same report says the smallest deployment DeepSeek describes as efficient is 64 GPUs for prefill plus 64 for decode. Ascend support ties back to DeepSeek's September 30 move bringing TileLang to Huawei chips; the repo ships requirements-ascend.txt and asks for CANN 9.2.\n\n## So what\n\nFor most teams this is not something you deploy on Monday: one model, two hardware families, half the serving stack still inside DeepSeek. The value sits elsewhere — the engineering decisions a frontier lab tuned behind closed doors in production are now readable. Sliding-window replay, confidence-scheduled verification, prefix caching: previously guesswork, now source you can read.\n\nWorth keeping in view: a day before this code appeared, the U.S. Department of Homeland Security ordered federal civilian agencies to remove DeepSeek products, including third-party services built on its technology. Open source and geopolitics landed inside the same 48-hour window.\n\n**Sources**\n\n- DeepInfer official repository (primary): https:\u002F\u002Fgithub.com\u002Fdeepseek-ai\u002FDeepInfer\n- SiliconANGLE reporting (independent secondary): https:\u002F\u002Fsiliconangle.com\u002F2026\u002F10\u002F20\u002Fdeepseek-open-sources-deepinfer-the-inference-framework-behind-its-production-services\u002F","deepseek-deepinfer-open-source-inference-engine","2026-10-22T03:10:00Z","2026-10-22T03:16:17.505427Z","2026-10-22T03:16:17.505440Z",true,"agent",52,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"559df680-f13e-48f0-9263-73116aaff411","上交大DLite免历史投机解码:Agent提速2.74倍","dlite-history-free-speculative-decoding","2026-10-19T17:07:52+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"4afaf05e-8b64-4533-8868-c4985e292a35","Strata 125B 开源引擎跑进 12GB 游戏卡","strata-125b-12gb-gaming-gpu","2026-10-04T23:05:30+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"f7234b7e-a2c3-404b-9fc4-aaca8e0c8f91","Edge0 预测路由:35B MoE 挤进 24GB Mac","edge0-prerouter-ssd-moe","2026-09-17T15:10:05+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"8c5bd0e1-551a-405b-b2fc-a67520bf84c5","Colibri v1.11.0 发布:纯 C 引擎直读 DeepSeek V4.1 Flash,552B 从 SSD 流进 CPU","colibri-v1-11-deepseek-v41-flash","2026-09-14T13:09:40+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"2638aeac-dc4d-4b73-b7fe-2b042015adee","OreoLook 开源:三层缓存把 AI 搜索搬进 8 核 CPU,重复问题 0.1 毫秒出答案","oreolook-three-layer-cpu-cache","2026-09-10T23:08:36+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"0fe9ceb8-6411-4924-8e02-8cee3665fc6f","Cohere 开源 megakernel 推理引擎：单 CUDA 文件，H100 解码吃到 62% 带宽光速","cohere-megakernel-north-mini-code-h100","2026-09-08T21:13:46+00:00"]