[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-abccd085-d910-4846-aae1-5f2136238f3b":3},{"id":4,"title":5,"summary":6,"original_url":7,"source_id":8,"tags":9,"published_at":23,"created_at":24,"modified_at":25,"is_published":26,"publish_type":27,"image_url":13,"view_count":28},"abccd085-d910-4846-aae1-5f2136238f3b","别怪大模型:Coding Agent 真正的瓶颈在「脚手架」——Queen's 35 版本控制变量实证","Coding Agent 的质量回退,工程社区第一反应往往是「模型又退步了」。但 Queen's University 的 Sghaier 等人在 arXiv:2607.03691 中指出:这是误诊。\n\n这是第一项把「脚手架」从 LLM 中剥离出来做控制变量的纵向研究。之前所有 SWE-bench 类工作都「固定脚手架、换模型」,本文反向操作——固定同一个底层 LLM,只换 scaffolding,看 35 个连续版本会怎样。\n\n研究覆盖 Codex、Qwen Code、Gemini、OpenCode、OpenHands 五大开源脚手架,先看生态:平均发布速度超过 2 次\u002F天,几个月累积几千个 issue。然后深挖 Qwen Code CLI 的 35 个顺序版本,每个版本都在 50 个分层抽样的 SWE-bench Verified 任务上跑分,全程锁定底层模型不变。\n\n结论令人警醒:任务成功率与效率的波动,绝大部分能追溯到具体 PR、具体架构组件,而不是底层模型。即使基座完全没动,一次 prompt 模板调整、一次工具执行顺序改动、一次上下文管理重构,都可能让质量曲线出现 5–10pp 量级的跳变。\n\n这篇论文的价值不在新方法,而在打脸一个普遍盲区:业界把算力、参数、榜单都押在模型上,却没人系统地度量脚手架本身的演化贡献。它呼吁把脚手架当成「一等公民」来监控——Agentic QA、Scaffolding Observability 应该像对待模型一样有 SLA、有回归测试。\n\n对正在自研 Coding Agent 的团队,这意味着:升级脚手架前必须做「控制变量基准」,否则你们会陷入「升一版差 5%、再升一版回来」的玄学循环。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2607.03691","7437aeb9-930c-4866-a2e9-48003c1a792b",[10,14,17,20],{"id":11,"name":12,"slug":12,"description":13,"color":13},"120fa59a-ff6f-4537-9bf5-f818df636a0e","benchmark",null,{"id":15,"name":16,"slug":16,"description":13,"color":13},"e82b2d09-81b2-43d1-977e-e018443b3c14","coding-agent",{"id":18,"name":19,"slug":19,"description":13,"color":13},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":21,"name":22,"slug":22,"description":13,"color":13},"c187600e-804c-4697-b828-1e4330e0eb10","qwen","2026-07-04T03:55:25Z","2026-07-12T00:12:11.884901Z","2026-07-12T00:12:11.884913Z",true,"agent",3]