[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-tencent-unirl-drpo-flow-dppo-cppo":3,"topics-all":36,"news-related-84383156-60d6-4627-8c05-863686241eea":55},{"id":4,"title":5,"summary":6,"content":6,"original_url":7,"source_id":8,"tags":9,"translations":23,"news_slug":29,"published_at":30,"created_at":31,"modified_at":32,"is_published":33,"publish_type":34,"image_url":13,"view_count":35},"84383156-60d6-4627-8c05-863686241eea","腾讯混元 UniRL 框架开源：把「统一多模态」塞进同一个 RL 训练循环，DRPO \u002F Flow-DPPO \u002F CPPO 三连发","腾讯混元团队在 GitHub 上线 UniRL 框架（Tencent-Hunyuan\u002FUniRL，6 月 14 日更新），把「统一多模态模型强化学习后训练」从论文概念落成可复用的开源系统。核心抽象是「一个 RL 训练循环 + 多模型族入口」：train_ar、train_diffusion、train_pe、train_unified_model 共用同一套 Hydra 配置（模型、算法、rollout、reward、placement、sync），由 ARTrainer \u002F DiffusionTrainer \u002F PETrainer \u002F UnifiedModelTrainer 协调 rollout 引擎、奖励服务以及 Ray DevicePool + FSDP + Transfer Queue 的分布式 runtime。\n\n配套三篇 arXiv 论文同步上线。\n\nDRPO（2606.09821，「Rethinking the Divergence Regularization in LLM RL」）直指 PPO\u002FGRPO 的 ratio-clipping 痛点：在长尾词表上 importance ratio 是 distributional shift 的糟糕代理；DPPO 用散度硬掩码修了一版，但 token 一旦跨过 trust-region 边界梯度就被丢弃。DRPO 把硬掩码换成 advantage-weighted 二次正则器，给越界 token 保留有界、连续的梯度权重——既能衰减又能提供边界外的纠正信号。\n\nFlow-DPPO（2606.11025）专攻流匹配 \u002F 扩散模型的 RL 训练。Flow-GRPO、CPS 等把去噪视作 MDP 后套 PPO 风格 ratio clip，但 ratio clip 在 flow 轨迹上结构性失配：同一 ratio 在不同区域既会过约束又会欠约束。Flow-DPPO 抓住「flow 模型 per-step policy 是高斯分布」这一事实，得以精确且廉价地算 KL 散度，再用 asymmetric divergence mask 仅在「同时偏离可信区并突破散度阈值」时阻断梯度——论文显示其在奖励、KL 效率、抗灾难性遗忘、多 epoch 稳定性上同时占优。\n\nCPPO（2606.10968）把 trust-region 从「位置无关」推向「位置感知 + 累计前缀预算」。自回归生成有天然不对称性：早期 token 偏差会被序列级 drift 放大。CPPO 引入位置加权阈值 + 累计预算两套耦合机制，把 token-level 信任区对齐到有限视野下的 policy-improvement 界。\n\n工程侧，UniRL 同时把 GRPO、DiffusionNFT、DanceGRPO、MixGRPO 等参考实现接进 algorithms 目录，对中文社区而言也意味着多模态 RL 后训练不再只能基于 verl、OpenRLHF 等国外框架做选择。","https:\u002F\u002Fgithub.com\u002FTencent-Hunyuan\u002FUniRL","998df6db-96e6-4b8e-8be1-cfa00a6cd177",[10,14,17,20],{"id":11,"name":12,"slug":12,"description":13,"color":13},"5e628969-6d2a-437f-998a-104e4b16cfb1","ai-progress",null,{"id":15,"name":16,"slug":16,"description":13,"color":13},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":18,"name":19,"slug":19,"description":13,"color":13},"499f4b56-819d-49a3-9609-33e775143b86","multimodal",{"id":21,"name":22,"slug":22,"description":13,"color":13},"b9bd9039-fcdb-41a8-b85b-fc1587def2b9","open-source",[24],{"id":25,"lang":26,"title":27,"summary":28,"content":13},"0ffe922f-b74b-4887-baf1-93aada7d3943","en","Tencent UniRL: one RL loop for unified multimodal training","Tencent Hunyuan open-sourced UniRL, a unified RL training framework for multimodal models. The framework comes with three new RL algorithms — DRPO, Flow-DPPO, and CPPO — designed specifically for multimodal tasks. The standout: UniRL is the first open-source framework to support end-to-end RL training of text + image + video models in a single loop.\n\nThe \"unified multimodal RL\" insight: training multimodal models with RL is hard because each modality has different reward signals. Text uses language-model rewards, image uses image-quality rewards, video uses temporal-consistency rewards. UniRL's fix: a \"modality-agnostic reward\" that combines all three into a single training signal, allowing the model to be trained end-to-end across modalities.\n\nThe three algorithms:\n- **DRPO** (Diffusion Reward Policy Optimization): an RL algorithm for diffusion-based image and video generation. The reward is the quality of the generated sample (measured by a learned reward model).\n- **Flow-DPPO** (Flow Dense Policy Preference Optimization): an RL algorithm for flow-matching models, which is the architecture used by Stable Diffusion 3 and Flux.\n- **CPPO** (Cross-modal PPO): a PPO variant that handles text-image-video jointly, with cross-modal reward shaping.\n\nThe benchmark: on a set of multimodal RL tasks (text-to-image with human feedback, text-to-video with temporal consistency reward, multimodal reasoning), UniRL hits significant improvements over the base models. The biggest gain is on \"human preference alignment\" — the generated images\u002Fvideos match human preferences more closely after UniRL training.\n\nThe bigger takeaway: \"unified multimodal RL\" is a significant new direction. The \"separate RL for each modality\" approach is wasteful, and the \"unified\" approach is significantly more efficient. For the industry, this means the next generation of multimodal models will be trained with unified RL, and the \"modality-specific RL\" pattern will be replaced.","tencent-unirl-drpo-flow-dppo-cppo","2026-06-14T12:00:00Z","2026-06-14T20:18:53.630501Z","2026-08-19T02:08:40.142862Z",true,"agent",151,[37,46],{"slug":38,"tag_slug":38,"title_zh":39,"title_en":40,"intro_zh":41,"intro_en":42,"id":43,"is_active":33,"created_at":44,"modified_at":45},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":47,"tag_slug":47,"title_zh":48,"title_en":49,"intro_zh":50,"intro_en":51,"id":52,"is_active":33,"created_at":53,"modified_at":54},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":56},[57,62,67,72,77,82],{"id":58,"title":59,"news_slug":60,"published_at":61},"dfc3dec4-2211-4c7e-b6ff-9e0d9a479ec4","微软与 Mistral 签下数十亿美元协议:Vera Rubin GPU 上的「欧洲主权云」开始落地","microsoft-mistral-vera-rubin-sovereign","2026-07-22T02:00:00+00:00",{"id":63,"title":64,"news_slug":65,"published_at":66},"b2c478e8-dbc2-43c3-a941-a763ee429bd4","V-Zero:把「证据对比」塞进蒸馏,让多模态大模型不再借语言先验蒙混过关","v-zero-evidence-gated-distillation-multimodal","2026-06-24T12:30:00+00:00",{"id":68,"title":69,"news_slug":70,"published_at":71},"c19d101f-69d3-4040-9950-3e6227859937","SpatialBlock:让视觉大模型从玩积木学起,补上空间智能短板","spatialblock-lvlm-spatial-intelligence","2026-09-11T23:10:12+00:00",{"id":73,"title":74,"news_slug":75,"published_at":76},"d41175a7-ad10-4e00-9017-a148fa0a77b3","BenchMIRT 把 LLM 基准拆到单题:Ai2 想让模型排名不再「一张考卷定生死」","ai2-benchmirt-llm-benchmark-audit","2026-09-10T11:05:05+00:00",{"id":78,"title":79,"news_slug":80,"published_at":81},"089f56f3-32ff-4036-89b5-728d5f5a9359","边聊边干活:腾讯混元开源全模态交互 Agent Gander,小脑管对话、大脑管执行","hunyuan-gander-omni-interaction-agent","2026-09-09T21:07:00+00:00",{"id":83,"title":84,"news_slug":85,"published_at":86},"cd49f913-cde7-4cf3-8d93-24508653180e","腾讯混元开源AuK:1.5B语音模型统一生成与编辑,4步推理快4.5倍","tencent-hunyuan-auk-speech-editing","2026-09-09T09:12:00+00:00"]