[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-yandex-aliceai-t5-sparse-moe":3,"topics-all":38,"news-related-21fe3c11-4ba4-4801-b6fc-60c4ae559dc1":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"21fe3c11-4ba4-4801-b6fc-60c4ae559dc1","Yandex 逆流开源:35B 参数的 T5 MoE,每个 token 只激活 0.6B","Yandex 开源搜索产品 Alice AI 的生产底座 AliceAI-T5-35B-A0.6B:encoder-decoder 加 512 专家稀疏 MoE,34.35B 参数每 token 仅激活约 0.6B。事实与 32K 长上下文基准压过 Qwen3.5-35B-A3B,但生产级优化推理不外放。","当所有开源大厂都在卷 decoder-only 的时候,Yandex 交了一份反向答卷:把搜索产品 Alice AI 的生产底座 AliceAI-T5-35B-A0.6B 开源到 Hugging Face。这是一个从零训练的 encoder-decoder(T5 风格)模型,总参数 34.35B,每个 token 只激活约 0.6B——上一次大团队认真做 encoder-decoder MoE 还是 2021-2023 年的 NLLB-MoE、Switch Transformer 和 FLAN-MoE。([HF model card](https:\u002F\u002Fhuggingface.co\u002Fyandex\u002FAliceAI-T5-35B-A0.6B))\n\n## 为什么要用\"过时\"架构\n\n答案藏在产品形态里。据 Yandex 工程博客披露,这是搜索页\"快速回答\"的生成模型:上游模型先检索文档,再用 80M 参数的小模型压缩成\"信息上下文\",最后由它生成答案——高峰期用户等的就是这几秒。每个 MoE 层挂 512 个专家、每 token 只路由 8 个。这种\"读长上下文、吐短答案\"的 RAG 场景,恰好是 encoder-decoder 的传统主场:encoder 一次性并行处理检索文档,比 decoder-only 的因果注意力更省算力;再叠上稀疏 MoE,容量按 35B 走,单 token 计算量却压到 0.6B 级。模型用 UL2 去噪目标在 15T token 上预训练,训练后段用 YaRN 把上下文从 8K 扩到 128K。([Habr 官方博客](https:\u002F\u002Fhabr.com\u002Fru\u002Fcompanies\u002Fyandex\u002Farticles\u002F1080654\u002F))\n\n## 跑分:不是全能,是偏科优等生\n\n官方 benchmark 表里,它对上的是 Qwen3.5 2B\u002F4B\u002F35B-A3B、Gemma 4 E4B、T5 Gemma 2 等 Base 模型。结论很清晰:事实性与抽取类全面领先——俄文事实基准 CultCat 68.0、WikiWebFacts 81.3,Ruler 32K 拿到 94.7,全部压过激活量 5 倍于己的 Qwen3.5-35B-A3B;但数学代码类(MATH 500 62.9、HumanEval 69.3)和 Ruler 128K(81.4)仍是 Qwen3.5-35B-A3B 更强。盲测 SbS 同样如此:胜过微调版 Qwen3.5-2B 77%、4B 54%、T5 Gemma 2.4B-4B 69%,唯独对 Qwen3.5-35B-A3B 只有 44%——官方也承认对方\"推理成本显著更高\"。\n\n## 训练细节里的两个信号\n\n一是训练稳定性:团队先用了 Macro-LBL 式辅助损失做专家均衡,结果在 15T token 规模下 encoder 侧路由直接崩了,最终换用 DeepSeek-V3 的 aux-free 路由(偏置项不走梯度)才稳住——MoE 均衡策略在大规模下的失效又添一例。二是优化器:他们选了 Muon 而非 AdamW\u002FAdan,称实验中表现更好。工程侧还有一条容易被忽略的免责:对外只开放 HF Transformers 推理,生产级优化推理仍在 Yandex 内部。官方数据显示该方案在线 A\u002FB 带来 +0.34% 的会话数增长,SbS 对 Google AI Overview 胜率 56.7%。\n\n对做检索增强或结构化抽取的团队,这是一个稀缺的非 decoder-only 参考点;但对通用基准党,它本来就不是为你准备的——专才架构的胜利,恰恰说明了\"模型设计跟着产品形态走\"这条老原则在 MoE 时代依然成立。","https:\u002F\u002Fhuggingface.co\u002Fyandex\u002FAliceAI-T5-35B-A0.6B","4222868a-e629-4d19-baae-70e0faafa6df",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"7e89b5cc-57db-4f37-bc6d-28919a73931c","model-release",{"id":19,"name":20,"slug":20,"description":14,"color":14},"d11f0044-8aef-487c-bebe-89ce4683a4a3","moe",{"id":22,"name":23,"slug":23,"description":14,"color":14},"b9bd9039-fcdb-41a8-b85b-fc1587def2b9","open-source",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"c27f6d46-a38f-48d2-828c-33fb6827e142","en","Yandex open-sources a 35B T5-style MoE with 0.6B active params","Yandex open-sources AliceAI-T5-35B-A0.6B, base of Alice AI answers: an encoder-decoder MoE, 34.35B params, 0.6B active, 15T training tokens.","While every major open-weight lab converges on decoder-only transformers, Yandex just shipped the opposite bet: it open-sourced AliceAI-T5-35B-A0.6B, the production backbone of Alice AI answers in Search, to Hugging Face. This is a from-scratch encoder-decoder (T5-style) model with 34.35B total parameters and only ~0.6B active per token — the last serious encoder-decoder MoE efforts from major teams (NLLB-MoE, Switch Transformer, FLAN-MoE) date back to 2021-2023. ([HF model card](https:\u002F\u002Fhuggingface.co\u002Fyandex\u002FAliceAI-T5-35B-A0.6B))\n\n## Why revive an \"outdated\" architecture\n\nThe answer sits in the product shape. According to Yandex's engineering blog, this model powers the \"quick answers\" card on the Search results page: upstream models retrieve documents, an 80M-parameter extractor compresses them into \"infocontexts\", and this model writes the final answer — all within seconds at peak load. That read-long-write-short RAG profile is exactly where encoder-decoder traditionally shines: each MoE layer holds 512 experts with top-8 routing per token, the encoder processes retrieved documents in parallel, cheaper than causal attention, while sparse MoE keeps capacity at the 35B level with per-token compute near a sub-billion model. It was pretrained with a UL2 denoising objective on 15T tokens, with context extended from 8K to 128K via YaRN late in training. ([Habr engineering blog](https:\u002F\u002Fhabr.com\u002Fru\u002Fcompanies\u002Fyandex\u002Farticles\u002F1080654\u002F))\n\n## Benchmarks: a specialist, not an all-rounder\n\nThe official table pits it against Qwen3.5 2B\u002F4B\u002F35B-A3B, Gemma 4 E4B, and T5 Gemma 2 base models. The pattern is crisp. It sweeps factuality and extraction: 68.0 on the Russian CultCat benchmark, 81.3 on WikiWebFacts, and 94.7 on Ruler 32K — all beating Qwen3.5-35B-A3B despite that model activating 5x more parameters per token. But math and code (MATH 500 at 62.9, HumanEval at 69.3) and Ruler 128K (81.4) still favor Qwen3.5-35B-A3B. Blind side-by-side evaluations tell the same story: 77% win rate over fine-tuned Qwen3.5-2B, 54% over 4B, 69% over T5 Gemma 2.4B-4B — but only 44% against Qwen3.5-35B-A3B, which the team itself notes is \"significantly more expensive at inference\".\n\n## Two training signals worth noting\n\nFirst, routing stability: the team started with a Macro-LBL-style auxiliary-loss balance, saw the encoder-side routing collapse at the 15T-token scale, and switched to DeepSeek-V3's aux-free routing (bias terms updated without gradients) to stabilize — another datapoint that auxiliary-loss balancing breaks at scale. Second, the optimizer: they chose Muon over AdamW and Adan, reporting better results in their experiments. One caveat easy to miss: external users get Hugging Face Transformers inference only; the optimized production inference stack stays inside Yandex. The company reports +0.34% sessions-per-user in online A\u002FB testing and a 56.7% SbS win rate against Google AI Overview.\n\nFor teams building retrieval-augmented or structured-extraction workloads, this is a rare non-decoder-only reference point. For general-benchmark watchers, it was never meant for you — a specialist architecture winning its niche is itself the lesson: model design still follows product shape, even in the MoE era.","yandex-aliceai-t5-sparse-moe","2026-09-16T19:11:43Z","2026-09-16T19:11:55.146524Z","2026-09-16T19:11:55.146534Z",true,"agent",53,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"8e730a3d-439b-45cf-961d-f77cf01469fd","Cohere 开源 218B 翻译专用 MoE:25B 激活,自测评分超 DeepL,2×H100 可部署","cohere-north-small-translate","2026-09-11T19:07:20+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"d941056b-c2e7-42e5-965a-a982c20b1169","Qwen3.8-Flash-Next 架构细节:Gated Residual 多分支残差 + QSA micro-block 稀疏注意力","qwen3-8-flash-next-cost-efficiency-architecture","2026-09-02T02:00:00+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"33f3b08b-c8a2-43ec-81cf-85e2b918f913","腾讯开源 Hy4 preview:770B MoE、1M 上下文,模型首次参与自身训练","tencent-hy4-preview-770b-moe","2026-08-29T15:00:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"3d36921f-3b84-4663-97a0-fee7d4eff795","汤森路透开源 Thomson-1.0-Small:持续学习改造 Qwen,3B 激活的 35B MoE","thomson-1-0-small-continual-learning","2026-08-28T19:10:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"804ab59a-a8d6-4b61-bf74-8f6f2bdae83c","智谱把 Flash 做成一件正经事:一次说清 GLM-5.3-Flash 的架构和 benchmark 真相","glm-5-3-flash-hybrid-attention-architecture","2026-08-27T08:00:00+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"7958a2f1-028c-4b4e-b134-0d5de9afc1c1","Motif 3 收官:韩国 314B MoE 改用 MIT 许可,从零起步架构首次面向商用","motif-3-mit-license-sovereign-ai","2026-08-24T00:00:00+00:00"]