[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-lara-mini-base-multilingual-moe":3,"topics-all":38,"news-related-a79651db-8451-4ee7-9091-809dff098c19":67},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"a79651db-8451-4ee7-9091-809dff098c19","英语只占41%:Translated开源多语MoE小模型","Translated公司开源Lara Mini base:7.2B总参、0.6B激活的MoE基座,6T token里非英语占2.18T、覆盖全部24种欧盟官方语言,128k上下文,CC BY-NC-SA 4.0许可。论文自报在36个不超过2B激活的开源基座中平均分57.0居首。","小模型变强的主流打法是「多读」:几十亿参数的模型动辄读十几万亿 token。但高质量英语文本就那么多,预算越大,越是在把同一批数据读了又读,收益递减。Translated 公司的 Lara 团队反着算了一笔账:与其把英语读到第 N 遍,不如把预算花在读得最少的数据——其他语言的文本上。\n\n## 行业底色:非英语数据占比不足 12%\n\n论文先盘了开源基座模型的家底:在他们综述的开源模型里,非英语预训练数据占比最高也只有 12%。专做欧洲语言的模型倒是给得多——Teuken-7B 的非英语份额有 51%,Apertus 是 40%,EuroLLM 是 37%——但论文指出,这类模型同规模下跑分明显落后于通用模型。行业里还有个老担忧:小模型容量有限,语言之间会互相挤占,也就是「容量竞争」。\n\nLara 团队的回应,是把数据配比和架构一起改。\n\n## 模型本体:7.2B 总参,0.6B 激活\n\nLara Mini base 从零预训练 6T token:英语只占 41%,44 种其他语言(覆盖全部 24 种欧盟官方语言,模型卡注明简体与繁体中文分开计数)合计 2.18T,占 36.4%。而且训练越往后非英语越多:英语占比从主阶段的 44% 一路降到退火阶段的 27%,其他语言从 46% 涨到 55%,在退火阶段反超英语,成为最大数据域。6T token 里 97% 是文档首读,几乎不靠重复堆量。\n\n架构同时为多语和推理成本设计:每一层都是细粒度 MoE,128 个路由专家每个 token 激活 4 个,外加 1 个共享专家;总参数 7.2B,激活只有 0.6B。34 层里仅 7 层保留全局注意力,其余由 Gated DeltaNet 和窗口 512 的滑窗注意力混排,推理状态不随输入变长而膨胀;再用 100B token 的第四阶段把上下文从 4,096 拉到 128k。注意这是基座模型:只会续写文本,没有指令微调,也没有安全微调。\n\n## 成绩单:57.0,论文自报口径\n\n在论文自己评测的 36 个激活参数不超过 2B 的开源基座模型里,Lara Mini base 四域平均 57.0,领先第二名 granite-4.0-h-tiny 3.1 分,多语平均 56.7,领先 Gemma-4-E2B 3.4 分,数学单项第一,英语第二,代码第五。代价侧的对照很直观:granite-4.0-h-tiny 与 Qwen3-1.7B 分别读了 22.5T 和 36T token,约为它的 4 倍和 6 倍;全部 56 个被比较模型里总分更高的只有 6 个,激活参数都是它的 4 到 14 倍。\n\n推理是另一半卖点:单张 A100、单人使用、不量化,生成速度约每秒 1,100 字符,与约 10 亿参数的稠密模型相当;所有总分更高的模型,生成速度至少慢 1.8 倍。最接近的 MiniCPM5-2B 总分只高 0.5 分,多语平均却低 8.7 分。\n\n## 语言在专家里怎么分家\n\n机制层的发现更有意思:论文用 Jensen–Shannon 散度度量每种语言在 128 个路由专家上的分布,结果显示语言在首层和末层路由差异明显,在中间层几乎一致——MoE 让不同语言「各走各的门」,论文认为这减少了语言间的容量竞争,正对着「多语互挤」的老担忧。\n\n## 所以呢\n\n权重已在 Hugging Face 以 CC BY-NC-SA 4.0 发布:研究和非商业使用免费,商用要另谈。团队还承诺发布 T-HellaSwag——把常识推理基准 HellaSwag 用机器翻译做成 110 个语言变体(105 种语言、25 种文字,每变体 200 篇、合计 22,000 篇),称质量优于现用的多语版本;截至论文提交,它仍是「将发布」状态。\n\n对做中文模型的读者,这篇的价值不在跑分榜,而在方法论:数据配比被当成一等设计变量来调,语言多样性不是要交的税,而是还没开采的矿。当英语语料的边际收益越来越薄,「下一个 token 从哪来」可能比「参数再堆多少」更值得算。\n\n论文:https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.16956\n权重:https:\u002F\u002Fhuggingface.co\u002Ftranslated\u002FLara-4-Mini-Base\n","https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.16956","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"7e89b5cc-57db-4f37-bc6d-28919a73931c","model-release",{"id":19,"name":20,"slug":20,"description":14,"color":14},"d11f0044-8aef-487c-bebe-89ce4683a4a3","moe",{"id":22,"name":23,"slug":23,"description":14,"color":14},"b9bd9039-fcdb-41a8-b85b-fc1587def2b9","open-source",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"4af94158-22f5-4488-80ff-efadbca0ea37","en","Open-sourced: a 7.2B MoE base model trained on 41% English","Lara Mini base is open-sourced: a 7.2B-total, 0.6B-active MoE, 6T tokens, 41% English. Per the paper it averages 57.0, first of 36 sub-2B open bases.","The mainstream recipe for stronger small models is to read more: models with a few billion parameters now routinely train on more than ten trillion tokens. But high-quality English text is finite, so growing budgets end up re-reading the same data with diminishing returns. The Lara team at Translated ran the arithmetic the other way: instead of reading English for the Nth time, spend part of the budget on the text a model has seen least — text in other languages.\n\n## The backdrop: non-English data below 12%\n\nThe paper first audits the field: across the open models it reviews, non-English pretraining data tops out at a 12% share. Models built for European languages give far more — Teuken-7B at 51%, Apertus at 40%, EuroLLM at 37% — yet, the paper notes, they score well below general-purpose models of the same size. A long-standing worry adds friction: in small models, languages compete for capacity.\n\nThe Lara team's answer changes the data mixture and the architecture together.\n\n## The model: 7.2B total, 0.6B active\n\nLara Mini base is pretrained from scratch on 6T tokens: English is only 41%, while 44 other languages — covering all 24 official EU languages, with Simplified and Traditional Chinese counted separately per the model card — total 2.18T, a 36.4% share. The balance shifts as training proceeds: English falls from 44% of the main stage to 27% of the anneal, while the other languages grow from 46% to 55% and outweigh English in the anneal. Of the 6T tokens, 97% are a first read of a document, so repetition barely props up the budget.\n\nThe architecture serves both multilinguality and inference cost. Every layer is a fine-grained MoE that activates 4 of 128 routed experts per token, plus one shared expert; total parameters are 7.2B, active just 0.6B. Only 7 of 34 layers keep global attention — the rest interleave Gated DeltaNet and sliding-window attention with a window of 512, so the inference state does not grow with input length. A fourth stage of 100B tokens then extends context from 4,096 to 128k. Note this is a base model: it continues text, with no instruction tuning and no safety tuning.\n\n## The scorecard: 57.0, self-reported\n\nAmong the 36 open base models with up to 2B active parameters that the paper evaluates itself, Lara Mini base averages 57.0 across English, multilingual, math and code, 3.1 points above granite-4.0-h-tiny; its multilingual average is 56.7, 3.4 points above Gemma-4-E2B; it is first on math, second on English, fifth on code. The cost contrast is stark: granite-4.0-h-tiny and Qwen3-1.7B read 22.5T and 36T tokens, about 4 and 6 times as many. Across all 56 compared models, only six score higher overall, all with 4 to 14 times its active parameters.\n\nInference is the other half of the pitch: on a single A100, single user, no quantization, it generates about 1,100 characters per second — on par with dense models of about 1B parameters — while every higher-scoring model generates at least 1.8 times more slowly. The closest, MiniCPM5-2B, is 0.5 points higher overall but 8.7 points lower on the multilingual average.\n\n## How languages split the experts\n\nThe mechanism finding may be the most interesting part: measuring, for each language, the Jensen–Shannon divergence between its distribution over the 128 routed experts and the mean over supported languages, the paper finds that languages route differently in the first and last layers and almost identically in the middle layers. MoE lets languages take different doors; the authors argue this reduces capacity competition between languages — a direct answer to the old worry.\n\n## So what\n\nWeights are on Hugging Face under CC BY-NC-SA 4.0: free for research and non-commercial use, commercial use to be negotiated separately. The team also promises T-HellaSwag, a machine translation of the HellaSwag commonsense benchmark into 110 language variants (105 languages, 25 scripts, 200 documents per variant and 22,000 in total), claiming higher quality than the commonly used multilingual version; at paper submission it was still marked as forthcoming.\n\nFor anyone building Chinese-language models, the value here is methodological: the data mixture was tuned as a first-class design variable. Language diversity is not a tax to pay but a mine not yet exhausted. As marginal returns on English corpora thin out, where the next token comes from may matter more than how many more parameters you stack.\n\nPaper: https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.16956\nWeights: https:\u002F\u002Fhuggingface.co\u002Ftranslated\u002FLara-4-Mini-Base\n","lara-mini-base-multilingual-moe","2026-11-14T19:30:00Z","2026-11-14T19:08:29.011108Z","2026-11-14T19:08:29.011116Z",true,"agent",253,[39,49,58],{"slug":40,"tag_slug":41,"title_zh":42,"title_en":43,"intro_zh":44,"intro_en":45,"id":46,"is_active":35,"created_at":47,"modified_at":48},"image-generation","text-to-image","图像生成：从文生图到逐框编辑的模型竞赛","Image Generation: from text-to-image to frame-level editing","文生图已是红海，战场正转向编辑与可控性：FLUX.2\u002F3 用开源和量化把生图搬进消费级显卡，Qwen-Image、Seedream 等国产模型紧咬不放，GPT Image 2 则把推理能力带进像素。本专题追踪图像生成与编辑模型的发布、架构演进、加速量化与商业化数据，看「好图」如何变成「可用的图」。","Text-to-image is a crowded race, and the frontier has moved to editing and control: FLUX.2\u002F3 push open weights and quantization onto consumer GPUs, Qwen-Image and Seedream close in from China, and GPT Image 2 brings reasoning into pixels. This topic tracks model releases, architecture shifts, acceleration work, and the business numbers behind \"pretty\" becoming \"usable\".","52de7c59-7864-4569-8ae7-3d656d26a1a1","2026-11-20T07:20:23.534506Z","2026-11-20T07:20:23.534527Z",{"slug":50,"tag_slug":50,"title_zh":51,"title_en":52,"intro_zh":53,"intro_en":54,"id":55,"is_active":35,"created_at":56,"modified_at":57},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":59,"tag_slug":59,"title_zh":60,"title_en":61,"intro_zh":62,"intro_en":63,"id":64,"is_active":35,"created_at":65,"modified_at":66},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":68},[69,74,79,84,89,94],{"id":70,"title":71,"news_slug":72,"published_at":73},"b6d8130b-f989-46a5-baa3-d81e66265ef8","Reflection开源Beam:501B MoE的RL账本","reflection-beam-501b-moe-rl-campaign","2026-11-20T13:30:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"554f6552-6c09-401a-99e3-c9ce60a497ca","ERNIE 5.5 基座:280B激活14B,对话版还没来","baidu-ernie-5-5-flash-base-open-weights","2026-11-05T12:00:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"32560a11-3d59-4397-8f01-e75f680a59db","月之暗面兑现承诺:K3.1 Pro 2.8T 权重开源上架","kimi-k3-1-pro-open-weights-huggingface","2026-11-04T23:07:35+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"4d0566f5-ef08-4c5f-befc-09dde8010620","Le Chonk 开源兑现:三周RL让TB冲上41%","mistral-large-4-open-weights-terminal-bench","2026-10-29T17:11:35+00:00",{"id":90,"title":91,"news_slug":92,"published_at":93},"03d33c23-e56a-41eb-8732-b0712059662f","PUBG 母公司开源 MoE:激活仅 4.3B 单卡可跑","krafton-raon-1-mini-preview-open-weights","2026-10-23T13:08:33+00:00",{"id":95,"title":96,"news_slug":97,"published_at":98},"b44fb160-9027-44cd-b355-a66869004abc","Nemotron 3.5 Super VL 开源:120B 总参只激活 12B","nemotron-3-5-super-vl-open-120b-moe","2026-10-20T17:11:56+00:00"]