[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-nous-tst-token-superposition-10b-2-5x":3,"topics-all":36,"news-related-1fe72544-eeb2-47ad-a849-450ef7056065":55},{"id":4,"title":5,"summary":6,"content":6,"original_url":7,"source_id":8,"tags":9,"translations":23,"news_slug":29,"published_at":30,"created_at":31,"modified_at":32,"is_published":33,"publish_type":34,"image_url":13,"view_count":35},"1fe72544-eeb2-47ad-a849-450ef7056065","Token-Superposition：10B MoE 预训练时长砍掉 2.5 倍","Nous Research 发布的 Token-Superposition Training（TST）是一种零侵入的预训练加速方法：在训练的前 r 步（r∈[0.2,0.4]），把 s 个连续 token 的 embedding 取平均作为一个 latent s-token，模型在 L\u002Fs 序列长度上预测下一个 bag 的 s 个目标 token，用多热交叉熵 MCE loss；之后切回标准 next-token 预测。该方法不动模型架构、optimizer、tokenizer、并行策略和数据，纯靠按 FLOPs 折算文本摄入量获得吞吐收益。3B dense 同 FLOPs 下 loss 由 2.808 降到 2.676，10B-A1B MoE 同 loss 下训练时长由 12,311 降到 4,768 B200-小时（约 2.5x 加速），HellaSwag、ARC-Easy\u002FChallenge、MMLU 全部反超。消融显示输入侧和输出侧是两种正交机制；论文也给出边界条件：仅适合算力受限、数据充足的预训练。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2605.06546","7437aeb9-930c-4866-a2e9-48003c1a792b",[10,14,17,20],{"id":11,"name":12,"slug":12,"description":13,"color":13},"7ac06d8e-b074-4147-abfc-ffaa4c6b8744","ai-efficiency",null,{"id":15,"name":16,"slug":16,"description":13,"color":13},"40269b40-7942-4650-9672-ed2e6524d37a","ai-technology",{"id":18,"name":19,"slug":19,"description":13,"color":13},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":21,"name":22,"slug":22,"description":13,"color":13},"b9bd9039-fcdb-41a8-b85b-fc1587def2b9","open-source",[24],{"id":25,"lang":26,"title":27,"summary":28,"content":13},"e7e8a018-2692-4def-ae92-78a8aba67288","en","Token-Superposition cuts 10B MoE pretraining time 2.5x","arxiv.org\u002Fabs\u002F2605.06546 introduces Token-Superposition Training (TST), a two-stage pretraining method from Nous Research. In the first stage, multiple input tokens are \"superimposed\" into a single embedding via averaging; in the second stage, the model is trained to \"de-average\" them, with the training time of a 10B MoE model compressed 2.5× and quality preserved.","nous-tst-token-superposition-10b-2-5x","2026-05-13T10:00:00Z","2026-06-05T22:20:24.956772Z","2026-08-19T02:08:40.142862Z",true,"agent",195,[37,46],{"slug":38,"tag_slug":38,"title_zh":39,"title_en":40,"intro_zh":41,"intro_en":42,"id":43,"is_active":33,"created_at":44,"modified_at":45},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":47,"tag_slug":47,"title_zh":48,"title_en":49,"intro_zh":50,"intro_en":51,"id":52,"is_active":33,"created_at":53,"modified_at":54},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":56},[57,62,67,72,77,82],{"id":58,"title":59,"news_slug":60,"published_at":61},"55663822-d49d-4c7e-a7c7-ba9770657c25","清华DBS动态分频:音源分离SI-SDR提升29%","dynamic-band-splitting-audio-separation","2026-10-21T15:02:40+00:00",{"id":63,"title":64,"news_slug":65,"published_at":66},"a491d4db-2d5a-4723-b093-28d1512fd647","VLA-Δ:物理状态变化当监督信号,LIBERO 98.4","vla-delta-physical-state-transition","2026-10-20T19:07:27+00:00",{"id":68,"title":69,"news_slug":70,"published_at":71},"ab1e6bdf-9433-41f0-be4d-cc0f78523ac2","SonicBoom v0.0.3:一个 .so 干掉手工调优","sonicboom-v003-mlir-compiler","2026-10-19T09:03:05+00:00",{"id":73,"title":74,"news_slug":75,"published_at":76},"50687317-89b7-4857-ad66-86e2e251ff9d","阿里开源 OpenGCUA：8 个前沿模型电脑操控平均涨 19.6 分","opengcua-open-computer-use-stack","2026-10-18T13:10:17+00:00",{"id":78,"title":79,"news_slug":80,"published_at":81},"36e4f28f-9176-485c-92b2-35cd0aac04fc","Apple MLX 联合 Meta 把 816B 模型装进 Mac Studio 集群","apple-mlx-muse-spark-mac-studio-thunderbolt-rdma","2026-10-16T03:00:00+00:00",{"id":83,"title":84,"news_slug":85,"published_at":86},"04ac054b-bc82-4832-9064-50a2defc33b2","Linus Torvalds 首批 LLM 辅助内核补丁进入审查","linus-torvalds-llm-assisted-folio-patches","2026-10-15T02:00:00+00:00"]