[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-modharness-rl-harness-evolves-with-policy":3,"topics-all":38,"news-related-2ad093e6-37e2-4982-a85e-9a3b6783ecf5":67},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"2ad093e6-37e2-4982-a85e-9a3b6783ecf5","权重在练,harness 也在练:脚手架成了可学习程序","arXiv 预印本 ModHarness-RL 把 agentic RL 手写固定的 harness 变成类型化块上的可学习程序:Compression 组装上下文,Steering 偏离时插指导,随 GRPO 共同演化。实测 CodeSearch 涨 14 到 24%,八个跨环境设置七个改善。","训练 agent 的强化学习有个不太起眼的怪状态:模型权重每一步都在变,但包在模型外面那层 harness——决定它能看到自己哪些交互记录、什么时候收到提示指导的脚手架——通常是人手写好,从第一步到最后一步一动不动。11 月 13 日,Zelin He 等八位来自宾州州立大学、LinkedIn 和加州大学圣巴巴拉的作者在 arXiv 放出预印本 ModHarness-RL,把这件事翻了出来:harness 不必是死的,它可以写成程序,从训练数据里学出来,再跟着模型一起演化。\n\n## 手写的 harness 撞上两个矛盾\n\n论文先摆问题。一个有效的 harness 要按任务、数据、模型和上下文预算裁剪,每种组合手工调一遍,不随规模扩展,所以它应该从数据里推导。更关键的是,有效 harness 在训练中一直在变:RL 自己会消化掉一部分失败模式,新组件只应该加给那些持续存在的失败——什么时候插手、什么时候放手,本身就是随训练动态变化的判断。\n\n作者梳理了近期的相关尝试:按预设时间表衰减固定脚手架、把脚手架重写与权重更新交替进行、从观察到的故障里编辑规划与记忆模块。按论文的归类,这些方法的更新空间要么预先固定,要么绑死在特定任务脚手架上,还缺一条从交互的类型化状态出发、与模型共同优化的通用路径。\n\n## 把 harness 写成类型化块上的程序\n\nModHarness-RL 分两层。表示层:交互被记录成结构化 State,一组带类型的块——每一轮的推理、工具调用、工具结果都是块。harness 的两个核心组件被写成读这些块的程序:Compression 程序在固定预算内把累积历史重新组装成模型上下文;Steering 程序检测行为偏离,条件命中时往上下文里追加一条指导。\n\n程序这个载体自带三个好处,论文的表述是:可学习——LLM 能直接从训练轨迹和对应奖励写出程序;计算便宜——每轮每条轨迹都跑得起,全程不用再调模型;可测试——每个候选 harness 都能在存档轨迹上回放,先评估再决定收不收。\n\n演化层是与 RL 交替的管线:从近期轨迹总结经验、提出候选程序、执行验证,再在留出的训练任务上把关。Steering 部分有个值得单独说的 Replay 机制:候选指导程序要在过去训练步的旧轨迹上重跑,只有随训练持续复发的失败才会被安装。论文给的两个判断案例很能说明问题:一类失败一直复发,judge 装上对应指导,失败被压下去;另一类失败本来就在变稀少,judge 按住不动,后续训练自己把它消掉了。\n\n## 数字:涨多少,省多少\n\n实验设置全部对齐:GRPO,200 步,单节点八张 H100;CodeSearch 用 Qwen3-4B 和 Qwen3-8B,BIRD SQL 用 Qwen3.5-2B 和 Qwen3.5-4B,ALFWorld 用 Qwen2.5-1.5B 和 Qwen2.5-7B;每种环境再分「预算中途耗尽」与「几乎不触顶」两种上下文档位。所有从轨迹学习的方法统一用 GPT-5-mini 当 creator。CodeSearch 任务是在真实仓库里定位 bug,训练用 SWE-Smith,评估在 SWE-Bench Lite、Verified、Pro。\n\n按论文实测:CodeSearch 上 ModHarness-RL 比固定 harness 基线相对提升 14 到 24%,对记忆、技能、提示词、harness 演化四类可学习基线的领先幅度是 5 到 12%;Qwen3-8B 受限预算一档整体 F1 报 0.412,相对基线涨 24%,充足预算一档 0.460,涨 14%。它在 CodeSearch 每个设置都超过 zero-shot 的 GPT-5-mini——也就是替它写程序的 creator——作者据此排除了「蒸馏 creator」的解释。ALFWorld 与 BIRD SQL 上,两个程序单独用,八个设置里七个超过基础 harness。\n\n效率侧更扎眼。预算充足时,Compression 让 ALFWorld 的 prefill token 少 74%、CodeSearch 少 10%,而且程序是确定性的,早期轮次渲染一致,前缀缓存大部分仍然有效。和 GPT-5-mini 在线压缩历史相比,4 个设置里 2 个分数更高、另外 2 个最多低 5%,但 teacher 调用与 token 按论文估算省约两个数量级,受限预算档每轮新增墙钟时间降幅按其估计超过千倍。\n\n消融部分有两个反直觉点。先练策略再优化 harness、或反过来,两种顺序都全面掉分——共演化本身有价值;Compression 只在基模型上拟合一次、训练中冻结,反而比每十步重拟合更好,Qwen3-8B 受限预算下后者掉到 0.279——上下文稳定对策略训练,比「更贴身」更重要。\n\n## 边界在哪\n\n论文自己的限定写得相当清楚:对比最充分的是 CodeSearch,另外两个环境主要验证程序泛化;候选程序的把关只跑在有限的留出任务上,偶尔会放进略差的程序;程序由 GPT-5-mini 这个专有模型写成——策略只看到程序执行输出,该文本不作为损失目标,程序建成后可复用;确定性程序只能捕捉显式规则可检测的模式,适合工具调用这类结构化、可验证的交互,对信号是语义性的开放式对话并不合适。规模也还小:1.5B 到 8B 的模型、单节点训练,离前沿训练现场有距离。\n\n## 所以呢\n\n这篇论文真正有意思的地方,是把「harness 工程」从手艺变成了可优化对象,而且优化回路便宜——程序不靠模型在线跑,回放存档轨迹就能打分。当模型的能力在训练中每天生长,harness 若还是三个月前那版手写脚本,错配只会越攒越大。反过来说,连「喂给模型什么上下文」本身都成了学习对象之后,agent 训练系统里剩下的纯人工环节,还有几个?\n\n引用:ModHarness-RL: Recasting Harnesses as Learnable and Evolving Programs for Agentic RL（https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.18533，arXiv 预印本,2026 年 11 月 13 日提交,尚未同行评审）\n","https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.18533","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"6ad31a14-c0da-42df-81fd-564281f768db","agentic-ai",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"5e628969-6d2a-437f-998a-104e4b16cfb1","ai-progress",{"id":19,"name":20,"slug":20,"description":14,"color":14},"e82b2d09-81b2-43d1-977e-e018443b3c14","coding-agent",{"id":22,"name":23,"slug":23,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"13cf81bc-582a-4db4-aca6-3b9b4fd10b56","en","ModHarness-RL: the harness trains alongside the model","ModHarness-RL turns the fixed agent harness into learned programs co-evolving with GRPO; CodeSearch gains 14-24%, 7 of 8 settings improve.","Agentic reinforcement learning runs in an odd state: the policy weights change at every step, while the harness wrapped around the model — the scaffold deciding what the model sees of its own interaction and when it receives guidance — is typically hand-written once and never touched again. On November 13, Zelin He and seven co-authors from Penn State, LinkedIn and UC Santa Barbara posted a preprint, ModHarness-RL, that calls this out: the harness does not have to be dead. It can be written as programs, learned from training data, and evolved alongside the model.\n\n## Two problems with a hand-written harness\n\nThe paper lays out the tension first. An effective harness must be tailored to the task, the data, the model and the context budget; hand-tuning one for every such combination does not scale, so it should be derived from data. More importantly, the effective harness keeps changing during training: RL resolves some failure modes on its own, so new components should be added only for failures that persist. When to step in and when to hold back is itself a judgment that shifts as training proceeds.\n\nThe authors survey recent efforts to adapt the harness during RL — decaying fixed scaffolds on a preset schedule, alternating scaffold rewrites with weight updates, editing planning and memory modules from observed faults. In their framing, these update spaces are fixed in advance or tied to a task-specific scaffold; what is missing is a general way to derive harness operations from the typed states of the interaction and optimize them with the model.\n\n## Harness as programs over typed blocks\n\nModHarness-RL has two layers. Representation: the interaction is recorded as a structured State, a set of typed blocks — each turn's reasoning, tool call and tool result are blocks. Two core harness components are written as programs over these blocks. A Compression program reassembles the accumulated history into the model's context under a fixed budget; a Steering program detects when behavior deviates and appends guidance when a condition fires.\n\nPrograms carry three properties, in the paper's own terms: learnable, since an LLM can write them directly from training trajectories and their rewards; computationally inexpensive, so they run on every turn of every trajectory without calling another model; and testable, so each candidate harness can be replayed over stored trajectories and scored before it is accepted or rejected.\n\nThe evolution layer is a pipeline alternating with RL: summarize experience from recent trajectories, propose candidate programs, verify them by execution, and gate them on held-out training tasks. The Steering side has a Replay mechanism worth singling out: each candidate guidance program is rerun over trajectories from past training steps, and only failures that persist as the model trains get an intervention installed. The two judge cases in the paper are telling. In one, a failure keeps recurring, the judge installs the guidance, and the failure is cut down. In the other, the failure is already rare and falling, the judge holds off, and continued training erases it on its own.\n\n## The numbers: how much, at what cost\n\nThe setup is held constant: GRPO, 200 steps, one node of eight H100s. CodeSearch uses Qwen3-4B and Qwen3-8B, BIRD SQL uses Qwen3.5-2B and Qwen3.5-4B, ALFWorld uses Qwen2.5-1.5B and Qwen2.5-7B, each under a context-limited and a context-sufficient budget. Every method that learns from rollouts uses GPT-5-mini as its creator. CodeSearch asks the agent to localize a bug in a real repository, training on SWE-Smith and evaluating on SWE-Bench Lite, Verified and Pro.\n\nPer the paper's own measurements: on CodeSearch, ModHarness-RL improves over the fixed-harness baseline by 14 to 24%, and leads the four families of learnable baselines (memory, skills, prompts, harness evolution) by 5 to 12%. With Qwen3-8B under a limited budget it reports an overall F1 of 0.412, up 24% over the base harness; under a sufficient budget, 0.460, up 14%. It also beats zero-shot GPT-5-mini — the very model that writes its harness programs — in every CodeSearch setting, which the authors take as evidence the gain is not distillation from the creator. On ALFWorld and BIRD SQL, each program applied alone improves over the base harness in seven of eight settings.\n\nThe efficiency side is sharper. Under a sufficient budget, Compression cuts prefill tokens by 74% on ALFWorld and 10% on CodeSearch, and because the program is deterministic, earlier turns keep an identical rendering and most of the prefix cache stays valid. Against online compaction by GPT-5-mini, Compression scores higher in 2 of 4 settings and at most 5% lower in the other two, while teacher-model calls and tokens drop by roughly two orders of magnitude over a training run, and the per-turn wall-clock overhead in the context-limited settings falls by more than a thousandfold in the paper's estimates.\n\nTwo ablations cut against intuition. Running the two stages sequentially — policy first then harness, or the reverse — reduces performance in every setting, so co-evolution itself carries value. And Compression, fit once against the base model and then frozen, beats refitting it every ten steps as the policy trains; with Qwen3-8B under a limited budget the refit variant drops to 0.279. A stable context matters more to policy training than a snugly updated one.\n\n## Where the edges are\n\nThe paper's own limitations are plainly stated: the comparison is most thorough on CodeSearch, while the other two environments mainly test whether each program generalizes; the gate scores candidates on a limited set of held-out tasks and can occasionally accept a slightly worse program; the programs are written by a proprietary model, GPT-5-mini — the policy sees only program output, that text is never a loss target, and the programs are built once and reused; and deterministic programs capture only patterns an explicit rule can detect, which suits structured, verifiable interaction like tool calls far better than open-ended conversation where the signal is semantic. The scale is also modest: models from 1.5B to 8B, a single node, some distance from frontier training runs.\n\n## So what\n\nThe genuinely interesting move here is turning harness engineering from craft into an optimizable object, with a cheap optimization loop — the programs do not run a model online, and replaying stored trajectories is enough to score them. As model capability grows daily during training, a harness frozen at the hand-written version from three months ago only accumulates mismatch. And once even 'what context to feed the model' becomes a learned object, how many purely manual decisions are left in an agent training system?\n\nReference: ModHarness-RL: Recasting Harnesses as Learnable and Evolving Programs for Agentic RL（https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.18533, arXiv preprint, submitted November 13, 2026, not yet peer-reviewed）\n","modharness-rl-harness-evolves-with-policy","2026-11-16T23:45:00Z","2026-11-16T23:10:41.402282Z","2026-11-16T23:10:41.402307Z",true,"agent",156,[39,49,58],{"slug":40,"tag_slug":41,"title_zh":42,"title_en":43,"intro_zh":44,"intro_en":45,"id":46,"is_active":35,"created_at":47,"modified_at":48},"image-generation","text-to-image","图像生成：从文生图到逐框编辑的模型竞赛","Image Generation: from text-to-image to frame-level editing","文生图已是红海，战场正转向编辑与可控性：FLUX.2\u002F3 用开源和量化把生图搬进消费级显卡，Qwen-Image、Seedream 等国产模型紧咬不放，GPT Image 2 则把推理能力带进像素。本专题追踪图像生成与编辑模型的发布、架构演进、加速量化与商业化数据，看「好图」如何变成「可用的图」。","Text-to-image is a crowded race, and the frontier has moved to editing and control: FLUX.2\u002F3 push open weights and quantization onto consumer GPUs, Qwen-Image and Seedream close in from China, and GPT Image 2 brings reasoning into pixels. This topic tracks model releases, architecture shifts, acceleration work, and the business numbers behind \"pretty\" becoming \"usable\".","52de7c59-7864-4569-8ae7-3d656d26a1a1","2026-11-20T07:20:23.534506Z","2026-11-20T07:20:23.534527Z",{"slug":50,"tag_slug":50,"title_zh":51,"title_en":52,"intro_zh":53,"intro_en":54,"id":55,"is_active":35,"created_at":56,"modified_at":57},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":59,"tag_slug":59,"title_zh":60,"title_en":61,"intro_zh":62,"intro_en":63,"id":64,"is_active":35,"created_at":65,"modified_at":66},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":68},[69,74,79,84,89,94],{"id":70,"title":71,"news_slug":72,"published_at":73},"da6cbb98-3b89-4631-8dbd-59411136fb30","哈佛用 700 家公司拆解 AI 编程智能体:30% 代码增量,49% 合并时延","harvard-coding-agent-review-bottleneck","2026-10-14T11:30:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"648ee35f-321e-4cc7-8e88-9a9d3ebae033","小模型每晚自学:一年后从 52 分掉到 5 分","on-device-llm-self-training-judge-failure","2026-11-19T03:30:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"bc983cc8-6c0e-4e7a-9d0c-cd6a3175c18e","模型学炼丹:350 条轨迹教会 GLM-5.3 训模型","rsi-prime-decision-sft-model-training","2026-11-18T15:30:00+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"b6285867-8ac1-4e91-bc05-d9f4a9ee7210","20个前沿模型做Excel,榜首只过38.3%","gdp-xlsx-excel-frontier-model-benchmark","2026-11-16T21:40:00+00:00",{"id":90,"title":91,"news_slug":92,"published_at":93},"6c8ae96a-53e0-4ef7-8609-40b345cdee7c","决策模型实测:自己演砸42.9%,教学生82.9%","decision-model-actor-judge-teacher","2026-11-15T21:45:00+00:00",{"id":95,"title":96,"news_slug":97,"published_at":98},"bd350599-7253-4f32-a874-9224bab89a49","多智能体搜索实测：32组对照里委派多数时候反而更差","multi-agent-search-delegation-accuracy","2026-11-15T09:10:00+00:00"]