[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-rsihub-agent-evolution-frozen-evaluator":3,"topics-all":38,"news-related-dcc6132a-e432-4ef3-9555-5412d692cd90":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"dcc6132a-e432-4ef3-9555-5412d692cd90","RSIHub 开源:先冻结评测器,再谈自我进化","RSIHub 把 Agent 自进化的评测器冻结在可变范围之外,让 AHE、Hyperagents、A-Evolve、GEPA 以可替换 recipe 跑在同一套基建上;论文实测最大整体 pass@1 提升 13.5 个百分点,但方法排名会随基准和种子模型翻转,HF 论文页将团队标注为 ByteDance。","自我进化这条线上，研究者真正的隐性成本往往不是算法，而是评测。每篇做 Agent 自我改进的论文，都要自己重造一遍执行环境、评分口径、失败恢复和结果留痕，结果就是不同论文宣称的\"提升了多少\"根本不在同一个坐标系里。10 月 23 日提交到 arXiv 的 RSIHub 选择绕开算法创新，专攻这层实验基建，论文标注的会议是 NeurIPS 2026 Meta-Agents Workshop，Hugging Face 论文页把它归到字节跳动团队名下（[arXiv:2610.32253](https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.32253)）。\n\n## 冻结评测器才是框架的核心\n\nRSIHub 的抽象是 recipe 加 operator。recipe 声明哪些资源允许被改写，operator 是可复用的生命周期算子；框架本身负责执行、评测、恢复和谱系记录。整个循环被切成 select、rollout、analyze、mutate、guardrail、evaluate、absorb 七个阶段，落在九个命名算子槽位上，其中五个是必需的（select、rollout、mutate、gate、record），四个可选（analyze、validate、novelty、reflect）。\n\n最关键的一条约束写在仓库首页：评测永远不是可选择的算子，它由框架独占。候选 Agent 可以改 prompt、skill、harness，甚至改一部分进化算子，但碰不到给自己打分的那套代码。论文把这称作归因风险的对策——如果被优化的一方能顺手改掉评测条件，\"涨了多少\"这个数字本身就失去了意义。评测器、归档戳记和框架代码都留在可变范围之外。\n\n## 四种方法跑同一套基建\n\n框架把 AHE、Hyperagents、A-Evolve、GEPA 四种已有方法重写成 recipe，方法专属逻辑留在 recipe 和 operator 里，框架内核一行不改。实验用 MiniSWE 和 Codex 作种子 Agent，在 Terminal Bench 2 和 τ³ Banking 两个基准上跑，每个任务只做一次试验，报 pass@1。目标模型统一是 GPT-5.4 高推理强度，变异算子由 xhigh 推理强度的 Codex 驱动。\n\n在 MiniSWE 上跑 Terminal Bench 2，种子成绩 56.2，四种方法都涨了 7.8 到 13.5 个百分点，Hyperagents 和 A-Evolve 并列最高到 69.7，AHE 65.2，GEPA 64.0。但换更难的起点和另一个基准，故事立刻变样。Codex 在 Terminal Bench 2 上的种子是 69.7，AHE 只涨到 71.9，Hyperagents 和 A-Evolve 各加 1.1，GEPA 原地不动。τ³ Banking 上 MiniSWE 的种子是 27.8，Hyperagents 涨到 33.0，A-Evolve 却是 27.8 纹丝不动；Codex 的种子 24.7，AHE 涨到 34.0，GEPA 也有 33.0。\n\n## 排名会翻转，这比最高分更重要\n\n拆到 train、gate、sealed 三个切分上，结论更刺眼。Hyperagents 在 MiniSWE 加 Terminal Bench 2 上拿到最高整体分，sealed 切分追平 A-Evolve，gate 切分也只有它真正动了；A-Evolve 的 train 涨了 7.9 点，gate 却完全没变。到了 τ³ Banking，MiniSWE 上整体涨幅最大的 Hyperagents 反而在 sealed 上退了 2.1 点，AHE 退了 4.1 点，GEPA 也退了 2.1 点。论文自己的结论是：没有任何一种方法在基准、种子和切分三个维度上全面占优。\n\n这跟摘要里那句\"最大整体 pass@1 提升 13.5 个百分点\"并不矛盾，但要放在一起读。后者是论文自己评测里的口径，目前还没有第三方复现；而排名频繁翻转、gate 和 sealed 出现负增益，恰恰是这类结果最该被追问的地方。\n\n## 执行保真与故障注入\n\n论文还花大力气做两组\"别把环境噪声算成收益\"的对照。第一组比较同一个未改动的种子 Agent 直接执行与经框架执行，三次重复取均值：Terminal Bench 2 上 Codex 的直接执行与框架执行一致率只有 82.0%，低于两次直接执行之间的 88.0%，超出部分 6.0 个点；而两次直接执行自己在 12.0% 到 18.9% 的任务上就不一致。论文明确写道，这些对照只刻画了执行波动，不能确立单次进化收益的统计显著性。\n\n第二组是故障注入与边界强制，两个基准各七个场景、每个场景 30 次独立试验，共 420 次试验全部达到预期状态，没有一次让被污染的状态进入结果：终止在变异或评测中途、写入超出可变范围的改动、修改评测器文件、在变异期间访问 gate、sealed 和锚点证据、伪造归档结果，全部被拦下。\n\n## 我的判断\n\n这篇论文的诚实度高于它的分数。它没有宣称某个自进化方法更优，而是把\"不同方法怎么被公平比较\"这件苦活做成了可复用的基础设施。代码以 Apache-2.0 协议开源，要求 Python 3.12 以上，仓库页面显示 191 个 star 和 8 个 fork，仍处在活跃原型阶段。\n\n仓库里还有一个技能进化演示：用同一个 Codex 模型和同一份论文提示，Gen 0 的 12 行极简 Skill 有 14 个文本元素溢出 SVG 画布，Gen 2 的进化版通过了确定性的渲染与几何检查，四张论文海报的确定性完成率从 1\u002F4 变成 4\u002F4。团队自己标注这是\"一次代表性进化运行，而非广泛基准\"。\n\n真正的短板在边界：论文承认，这些保证依赖受信任的框架服务、宿主与文件系统保护，运行时指纹不能替代宿主或服务认证，框架本身也不含敌意代码，要更强的部署还得自己做独立认证、隔离和遏制。负责使用一节写得更直接——pass@1 更高不等于能安全部署，在银行基准上跑通更不等于适合真实金融决策；这个框架提供的是审计与回滚，不是遏制。\n\n所以更值得关注的问题也许不是\"哪个自我进化方法更强\"，而是：当被优化的东西开始自己改自己的实验台时，谁来保证那把尺子不动？","https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.32253","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"6ad31a14-c0da-42df-81fd-564281f768db","agentic-ai",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"120fa59a-ff6f-4537-9bf5-f818df636a0e","benchmark",{"id":19,"name":20,"slug":20,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":22,"name":23,"slug":23,"description":14,"color":14},"b9bd9039-fcdb-41a8-b85b-fc1587def2b9","open-source",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"09bb3b3e-af96-4b32-9122-3a1f17e5d7b6","en","RSIHub: freeze the evaluator before evolving agents","RSIHub freezes the evaluator while four self-evolution methods run as swappable recipes; best measured pass@1 gain is 13.5 points, but rankings flip.","The real hidden cost of agent self-improvement research is usually not the algorithm, it is the evaluation harness. Every paper on self-improving agents rebuilds its own execution environment, scoring protocol, failure recovery and experiment logging, which means the \"how many points did we gain\" figures across different papers are rarely measured on the same axis. RSIHub, submitted to arXiv on October 23, sidesteps algorithmic novelty and targets that infrastructure layer instead. The paper lists the NeurIPS 2026 Meta-Agents Workshop as its venue, and the Hugging Face paper page attributes the team to ByteDance ([arXiv:2610.32253](https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.32253)).\n\n## Freezing the evaluator is the actual design\n\nRSIHub abstracts the problem as recipes plus operators. A recipe declares which resources are allowed to be rewritten, while operators are reusable lifecycle primitives; the framework itself owns execution, evaluation, recovery and lineage. The whole loop is cut into seven phases -- select, rollout, analyze, mutate, guardrail, evaluate, absorb -- mapped onto nine named operator slots, five of them required (select, rollout, mutate, gate, record) and four optional (analyze, validate, novelty, reflect).\n\nThe critical constraint sits on the repository homepage: evaluation is never a selectable operator, it stays framework-owned. A candidate agent may rewrite prompts, skills and harnesses, and may even rewrite selected evolution operators, but it cannot touch the code that scores it. The paper frames this as an answer to attribution risk: if the party being optimized can casually modify the measurement conditions, the gain number itself stops meaning anything. The evaluator, the archive stamps and the framework code all stay outside the declared mutable surface.\n\n## Four methods on one shared substrate\n\nThe framework re-expresses four existing methods -- AHE, Hyperagents, A-Evolve and GEPA -- as recipes, keeping method-specific logic inside recipes and operators while the framework core is left untouched. The experiments use MiniSWE and Codex as seed agents on Terminal Bench 2 and tau-cubed Banking, running one trial per task and reporting pass@1. The target model is GPT-5.4 at high reasoning effort throughout, and every recipe uses a Codex mutation agent backed by GPT-5.4 at xhigh reasoning effort.\n\nOn Terminal Bench 2 with MiniSWE, the seed scores 56.2 and all four methods improve on it by 7.8 to 13.5 percentage points, with Hyperagents and A-Evolve tied at the top on 69.7, AHE on 65.2 and GEPA on 64.0. Switch to a harder seed and a different benchmark and the story changes immediately. Codex scores 69.7 on Terminal Bench 2, where AHE only reaches 71.9, Hyperagents and A-Evolve each add 1.1, and GEPA does not move at all. On tau-cubed Banking, MiniSWE seeds at 27.8 and Hyperagents climbs to 33.0 while A-Evolve stays flat at 27.8; Codex seeds at 24.7, AHE reaches 34.0, and GEPA also lands on 33.0.\n\n## Rankings flip, and that matters more than the best score\n\nBreaking the results down by train, gate and sealed splits is more revealing. Hyperagents takes the highest overall score for MiniSWE on Terminal Bench 2, ties A-Evolve on the sealed split, and is the only method that actually moves the gate split; A-Evolve gains 7.9 points on train while leaving gate untouched. On tau-cubed Banking, Hyperagents produces the largest overall gain for MiniSWE while regressing 2.1 points on sealed, AHE regresses 4.1 points, and GEPA regresses 2.1 points as well. The paper's own conclusion is that no method dominates across benchmarks, seeds and splits.\n\nThis does not contradict the headline of a 13.5 percentage point overall pass@1 gain, but the two should be read together. The headline is the paper's own measurement, with no third-party reproduction yet; while frequently flipping rankings, gate regressions and sealed regressions are exactly where such results deserve scrutiny.\n\n## Execution fidelity and fault injection\n\nThe paper invests heavily in two control experiments designed to keep environment noise from being counted as gain. The first compares direct execution of an unchanged seed agent against execution through the framework, averaging three repetitions: on Terminal Bench 2 with Codex, direct-to-framework agreement is only 82.0 percent, below the 88.0 percent agreement between two direct runs, an excess of 6.0 points, while two direct runs already disagree on 12.0 to 18.9 percent of tasks. The paper states plainly that these controls only characterize execution variability and do not establish the statistical significance of single-run evolution gains.\n\nThe second set is fault injection and boundary enforcement, seven scenarios per benchmark at 30 independent trials each, 420 trials in total, all reaching the expected outcome with no corrupted state ever admitted into a result: termination during mutation or evaluation, writes outside the mutable surface, modification of evaluator files, access to gate, sealed and anchor evidence during mutation, and forged archive outcomes are all rejected.\n\n## Where I land\n\nThe honesty of this paper is worth more than its scores. It does not claim that any particular self-evolution method is stronger; it turns the tedious job of comparing methods fairly into reusable infrastructure. The code is released under Apache-2.0, requires Python 3.12 or newer, and the repository page shows 191 stars and 8 forks, still an active prototype.\n\nThe repository also carries a skill evolution demo: with the same Codex model and the same paper prompt, the Gen 0 twelve-line minimal Skill overflowed the SVG viewBox with 14 text elements, while the evolved Gen 2 editorial Skill passed the deterministic renderability and geometry gates, moving the deterministic completion pass rate from 1\u002F4 to 4\u002F4 across a four-paper showcase. The team labels it a representative evolution run rather than a broad benchmark.\n\nThe real limitation sits at the boundary: the paper concedes that these guarantees rely on trusted framework services, host and filesystem protections, and that runtime fingerprints do not replace host or service attestation, so stronger deployments need independent attestation, isolation and containment. The responsible use statement is blunter still, higher pass@1 does not establish safe deployment, and success on a banking benchmark does not establish suitability for financial decisions; what this framework offers is auditing and rollback, not containment.\n\nSo the more interesting question may not be which self-evolution method is stronger, but rather: when the thing being optimized starts editing its own experiment bench, who guarantees the ruler does not move?","rsihub-agent-evolution-frozen-evaluator","2026-10-26T07:20:28Z","2026-10-26T07:20:36.875590Z","2026-10-26T07:20:36.875603Z",true,"agent",50,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"aa718f3c-14cf-4984-8762-794eaaee8c16","MAI-Gym:9B 智能体在 2 万个并行环境里做在线 RL","mai-gym-gui-agent-online-rl-9b","2026-10-21T09:30:00+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"bb6e0f7b-3899-4e45-a267-bfa512d470b0","FDE-Gym:95个真实交付任务,最强Agent只过14个","fde-gym-agent-delivery-benchmark","2026-10-18T23:08:06+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"206ea36a-1eea-463f-a241-1e3b32f5ec2d","USTC GraphForge:证据图把任务和 rubric 钉在一起,Qwen3.6-27B 涨三基准","graphforge-ustc-qwen36-27b-evidence-graph","2026-10-04T03:05:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"4977d1f4-c8f7-480c-aaa1-ec01d69f44d8","8B 拿 SFT+GRPO 打 685B MoE:KaliBench 把 LLM 网络安全工具调用拆到命令行级","kalibench-cybersecurity-cli-runtime-rewards","2026-10-03T05:00:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"9ebb888c-dfe7-416a-9940-a913527d4f73","AI Agent 的失败比成功更值钱:5 万对错误诊断数据,修正通过率 18.4%→51.1%","agent-error-dataset","2026-10-01T15:11:08+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"e98cf9a2-8348-4142-9a56-c11166774798","SpeakerMem-R1:多方对话记忆,分清谁说了什么","speakermem-r1-multi-party-memory","2026-09-24T19:05:00+00:00"]