[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-inclusionai-ling-3-flash-hybrid-linear-moe-agent":3},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"259d91b2-ed6b-4af8-8f2a-f759b84cc617","Ling-3.0 Flash：蚂蚁 InclusionAI 用 KDA+MLA 混合线性注意力 + 1\u002F64 稀疏 MoE，把 124B\u002F5.1B 压成 Agent 生产级 Flash 模型","InclusionAI 于 8 月 2 日放出下一代原生混合推理模型 Ling-3.0 Flash：124B 总参 \u002F 5.1B 激活参数，5:1 交替堆叠 KDA + MLA，原生集成 SGLang HiCache + Mooncake 分层缓存（长输入 TTFT 降 60%–80%）。在 SWE-Bench Pro 56.6%、SWE-Bench Multilingual 72.4% 等 Agent 关键评测上对标 1T 级旗舰，4×H20 \u002F 4×Blackwell 即可起跑。已接入 Claude Code、Kilo Code、Hermes Agent、OpenClaw 等主流 agent harness。","# Ling-3.0 Flash:蚂蚁 InclusionAI 用「KDA+MLA 混合线性注意力」+ 1\u002F64 稀疏 MoE,把 124B 总参 \u002F 5.1B 激活参数压成 Agent 生产级 Flash 模型\n\n8 月 2 日,蚂蚁集团发起、定位为「开放科学型 AGI 实验室」的 InclusionAI,在 Hugging Face 上放出了下一代原生混合推理模型 **Ling-3.0 Flash**。它的核心叙事不是「再做一个更大参数的 MoE」,而是用一组相当激进的架构选择,把 **124B 总参数 \u002F 5.1B 激活参数** 的模型,做到推理速度、长上下文效率、Agent 工作流都可比肩 1T 级旗舰的水平。\n\n## 一组反直觉的工程取舍\n\n按照 InclusionAI 给出的参数表,Ling-3.0 Flash 在架构上有四个值得展开的点:\n\n- **混合线性注意力从预训练第一层就开始**,而不是「先训全注意力、再蒸馏出线性层」。模型采用 5:1 交替堆叠的 **Kimi Delta Attention(KDA)+ MLA**:35 层 KDA + 7 层 Gated MLA,再加 2 层 Dense,共 44 层。这种从一开始就走 hybrid-linear 的做法,目的是让线性注意力天然吃到预训练分布,而不是事后打补丁。\n- **KDA 细粒度对角门控 + 1\u002F64 稀疏 MoE**:512 个 routed expert,每 token 激活 8 个,再加 1 个 shared expert。换算下来路由稀疏度约 1\u002F64,这是当前主流 MoE 模型(1\u002F8 到 1\u002F16)之外更激进的一档。\n- **注意力头 32、Hidden 2560、专家中间层 768**:用偏小的 hidden \u002F 专家容量换「激活参数足够轻」,这是「能跑在 4×H20 \u002F 4×Blackwell」的关键。\n- **训练上下文从 8K → 32K → 256K 三段式 schedule**,而不是一步到位塞 256K。配合 SGLang 的 `dev-Ling-3.0-flash` 镜像,推理端可以用 1\u002F64 MTP(`NEXTN`)做投机解码。\n\n为了吃下长上下文场景,Ling-3.0 Flash 原生集成 **SGLang HiCache + Mooncake 分层缓存**(物理双池 + 集群共享 L3),官方称在长输入场景下能把 **TTFT(首 token 时延)降低 60%–80%**——这对 Deep Research \u002F BrowseComp 这类「开十几个页面、不停回灌历史」的任务,几乎是基础设施级的改进。\n\n## 评测表现:5.1B 激活参数打 1T 旗舰的「Key Bench」\n\nInclusionAI 给出的评测覆盖几类最容易被 Agent 框架卡脖子的任务:\n\n- **SWE-Bench Pro:56.6%**(同档位 128B 范围内第一档)——这是 OpenHands harness 下、256K 上下文、decoding 用 `temperature=0.6, top_p=0.95, max_new_tokens=32K` 的实测,直接衡量「拿到一个真实 GitHub issue 之后,能不能自己改完并跑通测试」。\n- **SWE-Bench Multilingual:72.4%**——多语言软件工程评测,对 Java\u002FJS\u002FPython 等主流语言 + 新功能 \u002F Bug Fix \u002F 重构多种开发场景的综合考验。\n- **MathArena AIME 2026:93.2、HMMT Feb 2026:87**——数学竞赛级任务。\n- **HLE(Humanity's Last Exam):22.7**——通用知识推理基线。\n- **Tau3-banking-AA、MCP-Atlas、SkillsBench**——分别对应金融域工具调用、MCP 工具全景、技能泛化。\n\n工程生态上,官方明确列出 Ling-3.0 Flash 已经吃下的框架:**Claude Code、Kilo Code、Qwen Code、Hermes Agent、OpenClaw**。换句话说,不是只能跑自家 demo,而是直接接入了当下最主流的 agent harness——这对一个开源 Flash 模型来说,比单卡跑分更有信号意义。\n\n## 为什么是「Flash」而不是「Pro \u002F Max」\n\nInclusionAI 在模型名里刻意保留「Flash」后缀,定位上和 Google 的 Gemini 3.7 Flash 类似:**不是旗舰本身,而是旗舰的快速版**。真正的 1T 级旗舰是 **Ring-2.6-1T**(对应 12.4 倍总参、约 8.1 倍激活参数);Ling-3.0 Flash 的目标是在 5.1B 激活参数这一档,做到「能进生产环境的 Agent 模型」。\n\n这种「同款架构不同规模」的 split,在 2026 年的开源 LLM 圈里已经形成一种共识:旗舰负责拉天花板评分,Flash \u002F Mini \u002F Lite 负责把成本、显存、TTFT 压到一个能塞进实际业务的水平。Ling-3.0 Flash 的 124B 总参 + 5.1B 激活 + 1\u002F64 稀疏,本质上是在「每个 token 算力」和「每个 token 记忆」之间找一个新的 sweet spot。\n\n## 推理部署成本:4 张 H20 \u002F 4 张 Blackwell 起步\n\nInclusionAI 给出的部署命令几乎是当前开源社区的「标准模板」:\n\n- **SGLang**:`docker pull lmsysorg\u002Fsglang:dev-Ling-3.0-flash`,然后 4×141GB 显存(H20-3e)或 4×Blackwell 节点,`--tp 4 --context-length 262144 --speculative-algorithm NEXTN --mem-fraction-static 0.8` 即可拉起。\n- **vLLM**:用 InclusionAI 维护的 `ling_3_0` 分支(`https:\u002F\u002Fgithub.com\u002FinclusionAI\u002Fvllm-ling-v3.git`),启用 MTP `--speculative-config '{\"method\":\"mtp\",\"num_speculative_tokens\":3}'`,4 GPU 起跑。\n- **OpenRouter**:Ling-3.0 Flash 在 OpenRouter 上有 `inclusionai\u002Fling-3.0-flash:free` 通道,直到 8 月 3 日前可免费调用;之后按 token 计费(具体定价 Hugging Face 模型卡未披露)。\n\n样本参数官方建议是 `temperature=0.6, top_p=0.95, top_k=20`,thinking 模式默认开启——通过 `chat_template_kwargs.enable_thinking=false` 关闭。这意味着 Ling-3.0 Flash 把它默认当做一个「思考型」模型来卖:不主动调 thinking 的开发者,会拿到思考后的最终答案;要快的话再手动关。\n\n## 我的判断:这条路线指向「Agent 时代的 Llama 3.1 8B」\n\n把 Ling-3.0 Flash 的几个数字摆在一起:124B \u002F 5.1B \u002F 1\u002F64 \u002F 56.6 SWE-Bench Pro \u002F 72.4 Multilingual \u002F SGLang 原生 \u002F HiCache \u002F MTP \u002F OpenRouter 免费——它实际上瞄准的是「Agent 时代的 Llama 3.1 8B \u002F Qwen2.5-Coder-7B」那个生态位:不大,但**性价比 + 开源 + 主流 agent harness 直接可用**。\n\n在中国 LLM 圈,这个生态位过去半年一直是 DeepSeek V4 Flash(284B 总参 \u002F 13B 激活)和阿里 Qwen3.6 Flash(35B A3B)占着的。Ling-3.0 Flash 的入场,等于把「1\u002F64 稀疏 + KDA 混合线性 + HiCache 长上下文」这套组合推到了开源社区面前——接下来值得看的不是 Ling 跑多高,而是 SGLang \u002F vLLM 社区会不会把这套 MTP + HiCache 模板化,以及 Kimi、Qwen、DeepSeek 会不会在下一轮迭代里跟进同样的「hybrid-linear + 1\u002F64」配方。\n\n如果跟进了,2026 下半年的开源 LLM 大概率会出现一个明显的拐点:**旗舰追 SOTA、Flash 拼 Agent 实用性**,两者开始走完全不同的优化目标。Ling-3.0 Flash 是这条路径上目前最明确的一颗信号弹。\n\n> 参考资料:\n> - 模型主页:[https:\u002F\u002Fhuggingface.co\u002FinclusionAI\u002FLing-3.0-flash](https:\u002F\u002Fhuggingface.co\u002FinclusionAI\u002FLing-3.0-flash)\n> - 官方主页:[https:\u002F\u002Fwww.inclusion-ai.org\u002F](https:\u002F\u002Fwww.inclusion-ai.org\u002F)\n> - SGLang 部署 Cookbook:[https:\u002F\u002Fdocs.sglang.io\u002Fcookbook\u002Fautoregressive\u002FInclusionAI\u002FLing-3.0-flash](https:\u002F\u002Fdocs.sglang.io\u002Fcookbook\u002Fautoregressive\u002FInclusionAI\u002FLing-3.0-flash)","https:\u002F\u002Fhuggingface.co\u002FinclusionAI\u002FLing-3.0-flash","e7a2ac45-ed74-4858-8663-3bd2943959a2",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"6ad31a14-c0da-42df-81fd-564281f768db","agentic-ai",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"7ac06d8e-b074-4147-abfc-ffaa4c6b8744","ai-efficiency",{"id":19,"name":20,"slug":20,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":22,"name":23,"slug":23,"description":14,"color":14},"b9bd9039-fcdb-41a8-b85b-fc1587def2b9","open-source",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"2c15cdfc-6656-4c6e-b0fc-d83f5a562826","en","Ling-3.0 Flash: How InclusionAI Used KDA + MLA Hybrid-Linear Attention and 1\u002F64 Sparse MoE to Make a 124B\u002F5.1B Agent-Grade Flash Model","On August 2, InclusionAI released Ling-3.0 Flash, a native hybrid-reasoning model with 124B total \u002F 5.1B activated parameters, a 5:1 alternating stack of KDA + MLA, and native integration of SGLang HiCache + Mooncake hierarchical caching (TTFT reduction of 60%–80% on long inputs). It scores 56.6% on SWE-Bench Pro and 72.4% on SWE-Bench Multilingual against 1T-class flagships, runs on 4×H20 \u002F 4×Blackwell, and plugs into Claude Code, Kilo Code, Hermes Agent, and OpenClaw.","# Ling-3.0 Flash: How InclusionAI Used KDA + MLA Hybrid-Linear Attention and 1\u002F64 Sparse MoE to Make a 124B\u002F5.1B Agent-Grade Flash Model\n\nOn August 2, 2026, **InclusionAI** — the open-science AGI lab founded by Ant Group — released the next-generation native hybrid reasoning model **Ling-3.0 Flash** on Hugging Face. The story is not \"another larger-parameter MoE,\" but rather a set of aggressive architectural choices that take a model with **124B total parameters \u002F 5.1B activated parameters** and put its inference speed, long-context efficiency, and agentic workflows on par with 1T-class flagships.\n\n## Four Counterintuitive Engineering Trade-offs\n\nAccording to InclusionAI's parameter sheet, four architectural choices in Ling-3.0 Flash deserve attention:\n\n- **Hybrid linear attention from the first layer of pretraining**, not \"full attention first, then distill linear layers later.\" The model uses a 5:1 alternating stack of **Kimi Delta Attention (KDA) + MLA**: 35 KDA layers + 7 Gated MLA layers, plus 2 Dense layers, for a total of 44 layers. Starting in a hybrid-linear regime from the very first pre-training step means the linear attention natively absorbs the pre-training distribution rather than acting as a post-hoc patch.\n- **KDA fine-grained diagonal gating + 1\u002F64 sparse MoE**: 512 routed experts, 8 activated per token, plus 1 shared expert. That routing sparsity (1\u002F64) is more aggressive than the current mainstream MoE range (1\u002F8 to 1\u002F16).\n- **32 attention heads, hidden size 2560, expert intermediate size 768**: smaller hidden \u002F expert capacity in exchange for \"activated parameters are light enough to run.\" This is what makes 4×H20 \u002F 4×Blackwell deployment possible.\n- **Three-stage training schedule: 8K → 32K → 256K context**, not \"stuff everything into 256K from day one.\" On the inference side, SGLang's `dev-Ling-3.0-flash` image uses 1\u002F64 MTP (`NEXTN`) for speculative decoding.\n\nTo absorb long-context workloads, Ling-3.0 Flash natively integrates **SGLang HiCache + Mooncake hierarchical caching** (physical dual-pools + cluster-shared L3). InclusionAI reports that in long-input scenarios, **TTFT (time to first token) is reduced by 60%–80%** — a near-infrastructure-level improvement for Deep Research \u002F BrowseComp style tasks that open dozens of pages and continuously reinject history.\n\n## Benchmark Performance: 5.1B Activated Parameters vs 1T Flagship on the Agent-Critical Tests\n\nInclusionAI's evaluation set focuses on the workloads that actually choke today's agent frameworks:\n\n- **SWE-Bench Pro: 56.6%** (top of the 128B-class tier) — measured under OpenHands harness with 256K context and decoding params `temperature=0.6, top_p=0.95, max_new_tokens=32K`. This directly measures: \"given a real GitHub issue, can the model fix it and pass tests?\"\n- **SWE-Bench Multilingual: 72.4%** — multilingual software engineering covering Java \u002F JavaScript \u002F Python and new-feature \u002F bug-fix \u002F refactor scenarios.\n- **MathArena AIME 2026: 93.2**, **HMMT Feb 2026: 87** — math competition level.\n- **HLE (Humanity's Last Exam): 22.7** — general-knowledge reasoning baseline.\n- **Tau3-banking-AA, MCP-Atlas, SkillsBench** — financial tool use, MCP tool coverage, skill generalization respectively.\n\nOn the engineering ecosystem side, the official model card lists the frameworks Ling-3.0 Flash already plugs into: **Claude Code, Kilo Code, Qwen Code, Hermes Agent, OpenClaw**. In other words, this is not just a self-demonstrated open-source model — it is integrated with the dominant agent harnesses in the wild, which is a stronger signal for a Flash-tier model than any single-card benchmark.\n\n## Why \"Flash\" and Not \"Pro \u002F Max\"\n\nInclusionAI deliberately kept the \"Flash\" suffix in the model name, similar in positioning to Google's Gemini 3.7 Flash: **not the flagship itself, but the fast variant of the flagship**. The real 1T-class flagship is **Ring-2.6-1T** (12.4× the total parameters, ~8.1× the activated parameters of Ling-3.0 Flash). Ling-3.0 Flash's goal is to deliver \"an Agent-ready model that can ship into production\" at the 5.1B-activated tier.\n\nThis \"same architecture, different scale\" split has become a quiet consensus in the 2026 open-source LLM world: the flagship chases SOTA, while Flash \u002F Mini \u002F Lite compress cost, memory footprint and TTFT down to a level that fits into real businesses. Ling-3.0 Flash's 124B total \u002F 5.1B activated \u002F 1\u002F64 sparsity is essentially looking for a new sweet spot between \"compute per token\" and \"memory per token.\"\n\n## Deployment Cost: 4× H20 or 4× Blackwell to Start\n\nThe deployment commands InclusionAI ships are essentially the standard template for the current open-source community:\n\n- **SGLang**: `docker pull lmsysorg\u002Fsglang:dev-Ling-3.0-flash`, then on 4×141GB VRAM (H20-3e) or a 4-GPU Blackwell node, run with `--tp 4 --context-length 262144 --speculative-algorithm NEXTN --mem-fraction-static 0.8`.\n- **vLLM**: Use InclusionAI's maintained `ling_3_0` branch (`https:\u002F\u002Fgithub.com\u002FinclusionAI\u002Fvllm-ling-v3.git`), enable MTP via `--speculative-config '{\"method\":\"mtp\",\"num_speculative_tokens\":3}'`, 4 GPUs minimum.\n- **OpenRouter**: Ling-3.0 Flash is exposed via `inclusionai\u002Fling-3.0-flash:free` on OpenRouter through August 3, 2026, after which it shifts to per-token pricing (specifics not disclosed on the model card).\n\nThe recommended sampling parameters are `temperature=0.6, top_p=0.95, top_k=20`, with thinking mode enabled by default — disable it per request with `chat_template_kwargs.enable_thinking=false`. The implication is that Ling-3.0 Flash ships as a \"thinking model\" by default: developers who do not explicitly toggle thinking get the post-think answer; turn it off when raw speed matters.\n\n## My Take: This Lineage Is Pointing at \"The Llama 3.1 8B of the Agent Era\"\n\nPutting Ling-3.0 Flash's numbers side by side — 124B \u002F 5.1B \u002F 1\u002F64 \u002F 56.6 SWE-Bench Pro \u002F 72.4 Multilingual \u002F SGLang-native \u002F HiCache \u002F MTP \u002F OpenRouter free tier — it is clearly aiming at the same ecosystem slot that \"Llama 3.1 8B \u002F Qwen2.5-Coder-7B\" once held: not the largest, but **the best price-performance + open-source + immediately compatible with mainstream agent harnesses**.\n\nIn the Chinese open-source LLM scene, that slot has been held over the past six months by DeepSeek V4 Flash (284B total \u002F 13B activated) and Alibaba's Qwen3.6 Flash (35B A3B). Ling-3.0 Flash's entry pushes the \"1\u002F64 sparse + KDA hybrid-linear + HiCache long-context\" combo into the open-source mainstream. The things to watch next are not how high Ling itself scores, but whether the SGLang \u002F vLLM community templatizes this MTP + HiCache recipe, and whether Kimi, Qwen, and DeepSeek adopt the same \"hybrid-linear + 1\u002F64\" formula in their next iteration.\n\nIf they do, the second half of 2026 will likely see a clear inflection in open-source LLMs: **flagships chase SOTA scores, Flash-tier models chase Agent practicality**, with the two optimizing for entirely different goals. Ling-3.0 Flash is, for now, the most explicit signal shot on that path.\n\n> References:\n> - Model page: [https:\u002F\u002Fhuggingface.co\u002FinclusionAI\u002FLing-3.0-flash](https:\u002F\u002Fhuggingface.co\u002FinclusionAI\u002FLing-3.0-flash)\n> - Official site: [https:\u002F\u002Fwww.inclusion-ai.org\u002F](https:\u002F\u002Fwww.inclusion-ai.org\u002F)\n> - SGLang deployment cookbook: [https:\u002F\u002Fdocs.sglang.io\u002Fcookbook\u002Fautoregressive\u002FInclusionAI\u002FLing-3.0-flash](https:\u002F\u002Fdocs.sglang.io\u002Fcookbook\u002Fautoregressive\u002FInclusionAI\u002FLing-3.0-flash)","inclusionai-ling-3-flash-hybrid-linear-moe-agent","2026-08-14T08:00:00Z","2026-08-14T00:07:51.056965Z","2026-08-14T00:07:51.056976Z",true,"agent",9]