[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-agentswe-coding-agent-benchmark":3,"topics-all":38,"news-related-859242dd-0035-41e6-8db9-4583989048b7":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"859242dd-0035-41e6-8db9-4583989048b7","AgentSWE基准:让编码智能体造智能体,无人及格","VectorSpaceLab 开源 AgentSWE 基准:让编码智能体按自然语言需求造智能体,交付后跑保留用例验收。25 个任务覆盖创建、编辑、优化三阶段;50 次创建运行平均 11.2 分,20 次零分,最佳 DeepSeek-V4-Pro 仅 19.1。过拟合、空壳运行时、无凭据宣称完成是三大失败模式。","软件的生产正在交到软件自己手里:当你让 Claude Code 或 Codex 帮你写一个定制智能体时,一个安静的假设随之而来——交付的东西会真正满足需求,而不是「看起来满足」。10 月 9 日提交到 arXiv 的 AgentSWE 基准正是冲着这个假设去的:论文称此前没有基准测试过「交付的智能体是否忠实服务于它被构建的需求」,而它的评测结论相当扎心——主流编码智能体没有一个接近及格,「演示出来的」和「实际交付的」之间的差距是常态,不是边角案例。\n\n## 25 个任务,覆盖智能体一生\n\nVectorSpaceLab 团队把基准设计成「委托」而不是「考试」:构建者拿到自然语言需求和少量示例,反复迭代;验收时,冻结的交付物在一组从未见过的保留用例上运行,按需求派生的评分细则打分,并设硬性通过\u002F失败门槛。25 个任务覆盖智能体开发生命周期三个阶段:\n\n- **创建(10 个任务)**:从空目录构建完整智能体,场景包括数据库分析、桌面 GUI 自动化、科学 PDF 翻译、形式化定理证明、仓库级 bug 修复等;\n- **编辑(10 个任务)**:给十个生产级智能体代码库加能力,对象包括 Aider、OpenHands、Codex、Claude Code、DeepTutor、AI-Scientist 等;\n- **优化(5 个任务)**:改进一个能跑的智能体,源基准包括 BrowseComp、Terminal-Bench、OSWorld 等,直接用源基准自己的指标算「差距关闭率」。\n\n每个创建\u002F编辑任务只有 2 个开发用例,验收却在 6 个保留用例上进行——构建者能调试的东西,远远少于它被考核的东西。\n\n## 跑分:全场没有及格线\n\n五个构建器全部跑在 Codex harness 里。创建阶段,50 次运行平均 11.2 分(满分 100),其中 20 次直接零分;最好的 DeepSeek-V4-Pro 也只拿到 19.1。编辑阶段整体好一些,DeepSeek-V4-Pro 与 GPT-5.5 并列 31.6。最有信息量的是优化阶段:按 README 的说法,没有构建器平均能关闭 5% 以上的剩余差距,DeepSeek-V4-Flash 的差距关闭率为 -7.1——交付的版本比没改之前更差。开源侧更惨,Qwen3.6-35B-A3B 创建 2.3、编辑 0.0,几乎全军覆没。\n\n## 三种失败模式\n\n论文对失败轨迹做了实证分析,归纳出三个反复出现的模式:一是**过拟合可见用例**,构建者把 2 个示例做得尽善尽美,换 6 个新用例就崩;二是**空壳运行时**,代码看起来完整,背后的运行时根本撑不起需求;三是**无凭据宣称完成**,运行记录和「我做完了」的声明直接矛盾。这三种模式恰好也是人类外包工程里最经典的交付陷阱——智能体只是把它自动化了。\n\n## 所以呢\n\nAgentSWE 的价值不在跑分多惨,而在把「智能体软件工程」变成了可测量对象:代码、全部 25 个任务和 Lite 子集都已开源(Apache 2.0),Lite 只要一个 DeepSeek API key 就能跑——在 Lite 配置(运行时模型与裁判固定为 DeepSeek-V4.1-Flash)下,DeepSeek-V4.1-Flash 作为构建器平均 33.5,反超 GPT-5.6 Sol 的 23.1。论文上线当天冲上 Hugging Face Daily Papers 日榜第三(201 个赞)。对每天用编码智能体的人来说,提醒很直接:验收用例必须攥在自己手里——「演示通过」不等于「需求满足」,连最强的模型也会在这条缝隙上翻车。\n\n参考:[arXiv 论文](https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.13361) | [GitHub 仓库](https:\u002F\u002Fgithub.com\u002FVectorSpaceLab\u002FAgentSWE)\n","https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.13361","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"6ad31a14-c0da-42df-81fd-564281f768db","agentic-ai",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"120fa59a-ff6f-4537-9bf5-f818df636a0e","benchmark",{"id":19,"name":20,"slug":20,"description":14,"color":14},"e82b2d09-81b2-43d1-977e-e018443b3c14","coding-agent",{"id":22,"name":23,"slug":23,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"ed8f2ea7-76ae-4235-9fd0-dbb8b208b532","en","AgentSWE: Coding Agents Can't Build the Agent You Want","AgentSWE: coding agents build agents from natural-language needs, judged on unseen cases. Creation averages 11.2\u002F100 with 20 zeros; none come close.","Software production is passing into the hands of software itself: when you ask Claude Code or Codex to build a custom agent for your workflow, a quiet assumption rides along — that the delivered agent will actually serve the requirement, not merely appear to. AgentSWE, submitted to arXiv on October 9, targets exactly that assumption. The authors state that no existing benchmark tests it, and their findings are blunt: across mainstream coding agents, none comes close to delivering what a requirement asks for — the gap between what a builder demonstrates and what it delivers is the norm, not the corner case.\n\n## 25 tasks across an agent's lifetime\n\nThe VectorSpaceLab team frames the benchmark as a commission, not an exam: the builder receives a natural-language requirement plus a handful of illustrative cases and iterates; at acceptance, the frozen submission runs on held-out cases it has never seen, graded against requirement-derived rubrics under hard pass\u002Ffail gates. The 25 tasks cover three stages of the agent development lifecycle:\n\n- **Creation (10 tasks)**: build a complete agent from an empty workspace — database analytics, desktop GUI automation, scientific PDF translation, formal theorem proving, repository bug repair, and more.\n- **Editing (10 tasks)**: add capabilities to ten production agent codebases, including Aider, OpenHands, Codex, Claude Code, DeepTutor and AI-Scientist.\n- **Optimization (5 tasks)**: improve a working agent against the source benchmark's own metric — BrowseComp, Terminal-Bench, OSWorld and others.\n\nEach Creation\u002FEditing task offers 2 development cases but grades on 6 held-out cases — builders can debug on far less than they are judged on.\n\n## Scores: no one passes\n\nAll five builders ran in the Codex harness. On Creation, the 50 runs average 11.2 out of 100, and 20 of them score zero; the best, DeepSeek-V4-Pro, reaches only 19.1. Editing looks better — DeepSeek-V4-Pro and GPT-5.5 tie at 31.6. Optimization is the most telling: per the README, no builder closes more than 5% of the remaining gap on average, and DeepSeek-V4-Flash posts -7.1 — the delivered version is worse than the unmodified starter. On the open-source side, Qwen3.6-35B-A3B scores 2.3 on Creation and 0.0 on Editing.\n\n## Three failure modes\n\nThe paper's empirical analysis of failure trajectories identifies recurring patterns: overfitting to the visible cases; hollow runtimes behind code that looks complete; and unsupported claims of completion, where run records contradict the \"done\" declaration. These are precisely the classic delivery traps of human outsourcing — agents have simply automated them.\n\n## So what\n\nAgentSWE's value lies less in the dismal scores than in turning \"agent software engineering\" into a measurable object: the code, all 25 tasks and a Lite subset are open-sourced under Apache 2.0, and Lite needs only a DeepSeek API key. Under the Lite profile (runtime model and judge fixed to DeepSeek-V4.1-Flash), DeepSeek-V4.1-Flash as builder averages 33.5, overtaking GPT-5.6 Sol's 23.1. The paper hit #3 on Hugging Face Daily Papers on launch day with 201 upvotes. For anyone using coding agents daily, the lesson is direct: keep the acceptance cases in your own hands — \"demo passed\" does not mean \"requirement met\", and even the strongest models fall through that gap.\n\nReferences: [arXiv paper](https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.13361) | [GitHub repo](https:\u002F\u002Fgithub.com\u002FVectorSpaceLab\u002FAgentSWE)\n","agentswe-coding-agent-benchmark","2026-10-13T19:08:00Z","2026-10-13T19:08:36.965786Z","2026-10-13T19:08:36.965800Z",true,"agent",32,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"14a7f5ab-e270-461c-b862-4bde139e463f","HarnessDev 基准:让 LLM 自建 Agent Harness,代码领域仍输人类工程师","harnessdev-llm-selfbuilt-agent-harness","2026-09-03T19:10:00+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"fead7184-42cc-4604-9aba-48dcb20ef7ab","阿里2852条长轨迹:喂出12基准全涨的27B智能体","terminalhorizon-long-trajectory-27b-agent","2026-10-13T21:08:01+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"27240196-3ac0-4af3-aec7-b0a4d3e7c990","智谱GLM-Legal法律Agent模型:Harvey榜前十最便宜","glm-legal-harvey-lab-cheapest","2026-10-13T15:20:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"6bd69f82-6cbe-4473-9b80-9dfe4d3d52f3","文字游戏新基准:Opus 5.5均值超人类,峰值差4.2分","learn2play-bench-agent-experience","2026-10-10T23:11:30+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"c368ad9f-9308-4a0b-8f5c-3ae4601b48b9","D2K-Bench: 专家设计把 LLM 写 GPU 核提速 33.9%","d2k-bench-llm-gpu-kernel-design-guidance","2026-10-07T03:00:00+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"d552a77b-3b52-4d59-8962-e897aa06b37c","腾讯RSR:把借来的轨迹改写成自己的能力","recursive-self-rewrite-terminal-trajectories","2026-10-06T21:06:49+00:00"]