[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-appo-ustc-alibaba-branching-agent-rl":3,"news-related-442a8bc3-60f2-40c9-826e-4683b289df2a":36},{"id":4,"title":5,"summary":6,"content":6,"original_url":7,"source_id":8,"tags":9,"translations":23,"news_slug":29,"published_at":30,"created_at":31,"modified_at":32,"is_published":33,"publish_type":34,"image_url":13,"view_count":35},"442a8bc3-60f2-40c9-826e-4683b289df2a","APPO：把 Agent RL 的分支点找准，LLM 智能体训练的细粒度新思路","APPO 这个工作我看到的第一反应是：终于有人把\"该在哪儿分支\"和\"该奖励谁\"这两件事拆开研究了。Agentic RL 过去半年进步很快，但主流方法基本还在用\"工具调用边界\"或\"固定工作流\"作为信用分配的颗粒度，这其实相当粗。\n\n论文的关键观察很犀利：作者通过 pilot 分析发现，influential decision points 实际上分散在整个生成序列里，而不是集中在 tool call 附近；与此同时，单看 token entropy 也无法可靠反映某个位置对最终结果的影响。换句话说，\"在工具调用处切一刀\"这种朴素的 branch 策略，以及\"高熵位置就是关键决策\"这种直觉，两件事都不成立。\n\n基于这个观察，APPO 提出了两个关键设计：其一是 Branching Score，把 token uncertainty 与 policy-induced likelihood gains 结合来挑选分支点，过滤掉那些熵高但实际无意义的位置；其二是 procedure-level advantage scaling，把 branched rollout 之间的 credit 分配做得更细致。在 13 个 benchmark 上，APPO 相比已有强基线稳定高出近 4 个点，同时还能保持 tool-call 效率与行为可解释性，没有靠堆 rollout 换分数。\n\n值得一提的是，APPO 是中科大与阿里合作的工作，代码已开源 (github.com\u002FAMAP-ML\u002FAPPO)。这种\"细粒度程序级 RL\"的方向，实际上与 agent 训练从 SFT 走向 RL 的范式转移是一致的：当模型从\"学会调用工具\"演化到\"多步长期规划\"，信用分配的粒度必须跟上，否则再多 rollout 也是浪费。我个人预期这套方法会成为后续 agentic RL 工作的标配基线之一，也值得所有做 Agent 训练的同学认真对照。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2606.12384","7437aeb9-930c-4866-a2e9-48003c1a792b",[10,14,17,20],{"id":11,"name":12,"slug":12,"description":13,"color":13},"6ad31a14-c0da-42df-81fd-564281f768db","agentic-ai",null,{"id":15,"name":16,"slug":16,"description":13,"color":13},"5e628969-6d2a-437f-998a-104e4b16cfb1","ai-progress",{"id":18,"name":19,"slug":19,"description":13,"color":13},"120fa59a-ff6f-4537-9bf5-f818df636a0e","benchmark",{"id":21,"name":22,"slug":22,"description":13,"color":13},"01598627-1ea6-4b27-a5d8-874971571a71","llm",[24],{"id":25,"lang":26,"title":27,"summary":28,"content":13},"3d62f3f5-70fd-4ae7-8902-fd10df0cb3c5","en","APPO finds the right branch points in agent RL training","arXiv 2606.12384 introduces APPO (Action-Process Policy Optimization), a fine-grained RL method for LLM Agents that focuses on \"branching points\" — the moments in a trajectory where the Agent must make a key decision. The result: 2-3× sample efficiency over standard PPO for Agent training.\n\nThe \"branching point\" insight: in a long Agent trajectory, only a small fraction of steps are \"branching points\" — moments where the Agent's decision significantly affects the outcome. Most steps are \"routine\" — the Agent follows a pre-determined path. Standard PPO treats all steps equally, wasting compute on the routine steps.\n\nThe APPO fix: APPO identifies branching points via a \"counterfactual analysis\" — for each step, the model estimates how much the outcome would change if a different action were taken. Steps with high counterfactual impact are \"branching points\" and receive more RL attention; steps with low impact are deprioritized.\n\nThe benchmark: on a set of Agent tasks (WebShop, ALFWorld, ToolBench), APPO achieves 2-3× better sample efficiency than PPO. The same number of training trajectories produces significantly better final performance. APPO is also more stable — it avoids the \"probability spike\" collapse that affects standard PPO.\n\nThe bigger takeaway: \"fine-grained RL\" is the right approach for Agent training. The \"all steps equal\" assumption of standard PPO is wasteful, and the \"focus on branching points\" approach is significantly more efficient. For the industry, this means Agent training pipelines should adopt APPO-style methods, and the next round of Agent RL research will focus on \"decision-point identification.\"","appo-ustc-alibaba-branching-agent-rl","2026-06-17T14:00:00Z","2026-06-17T14:11:24.587253Z","2026-08-19T02:08:40.142862Z",true,"agent",141,{"items":37},[38,43,48,53,58,63],{"id":39,"title":40,"news_slug":41,"published_at":42},"2035a9c7-2bc8-404d-9646-1813cbe4fa30","腾讯混元 LHTB：长程终端 Agent 最强仅 15.2% pass@1","tencent-hunyuan-lhtb-benchmark","2026-07-15T00:00:00+00:00",{"id":44,"title":45,"news_slug":46,"published_at":47},"c8684f9e-1028-4b81-a530-b04b0e044d3b","DiscoBench：搜得越多反而越错？首个聚焦\"何时该向用户问清楚\"的搜索 Agent 基准","discobench-clarify-search","2026-07-05T10:15:00+00:00",{"id":49,"title":50,"news_slug":51,"published_at":52},"75908d93-a928-4695-b966-d98847d135cb","SkillComposer 把 Agent 的技能选择重做成技能序列生成:GPT-5.2-Codex 提升 +23.1pp","skillcomposer-skill-sequence","2026-07-01T04:15:00+00:00",{"id":54,"title":55,"news_slug":56,"published_at":57},"9bfd8a69-2c97-40b7-9980-1e183fa61892","\"ALE 把 Agent 拽到真实工单前：1,490 道行业任务，主流配置通过率仅 2.6%\"","ale-agents-last-exam-1490-2-6-pct","2026-06-26T08:15:00+00:00",{"id":59,"title":60,"news_slug":61,"published_at":62},"685136b8-82ac-4deb-a81b-b47109c5056b","Open Agent Leaderboard 把评测对象从模型换成 Agent 系统:同一模型为何能跑出三个分数","open-agent-leaderboard-ibm-hf-agent-system","2026-06-23T12:01:00+00:00",{"id":64,"title":65,"news_slug":66,"published_at":67},"7b1b1217-91db-42c0-9467-fb6e45762d26","用「预测有效性」取代「平均分」:IBM 等 14 家伙伴给 LLM Agent 评测立下新规矩","ibm-14-partner-predictive-validity-agent-eval","2026-06-20T20:01:00+00:00"]