[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-fde-gym-agent-delivery-benchmark":3,"topics-all":38,"news-related-bb6e0f7b-3899-4e45-a267-bfa512d470b0":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"bb6e0f7b-3899-4e45-a267-bfa512d470b0","FDE-Gym:95个真实交付任务,最强Agent只过14个","新基准 FDE-Gym 让 AI Agent 当前置交付工程师:在 95 个源自真实项目、覆盖 18 行业的客户场景里,问出没人写下的约束、直接改类生产系统,按客户业务指标打分。13 种配置里最好成绩平均仅 0.56,95 个任务只干净完成 14 个;拿到全部关键事实的任务里也仅 10 个达标。","现有 agent 基准有个共同前提:任务定义是完整的——issue 报告、规格文档、任务工单,agent 照着做就行。真实世界的客户交付不是这样:需求模糊,约束没人写下,系统正在运行。10 月 15 日提交到 arXiv 的 FDE-Gym 基准正是冲着这个缺口来的,论文页上线即冲上 Hugging Face 日榜,拿到 285 个 upvote。\n\n## 基准怎么设计\n\nFDE-Gym 由 DeepLethe 团队的 26 位作者构建,包含 95 个端到端交付任务,来自 20 个客户场景、覆盖 18 个行业,每个任务都由行业专家从真实交付项目改编,领域横跨银行、医疗、物流、制造、能源等。Agent 被放进前置交付工程师(FDE)的位置:在模拟客户环境里与脚本化的干系人交互——发消息、试运行、等待都要扣分——并在类生产系统上改出能用的方案。\n\n评分不看 agent 做了什么,只看客户的数字变好多少:归一化到 0-1,0 是什么都没改,1 是专家参考解。GitHub 仓库(开源,Apache 2.0)给出更直观的口径:百分制下 80 分才算通过,生产出事故直接归零。\n\n## 结果:信息不是瓶颈,交付才是\n\n团队评了 7 家厂商的 13 种模型配置。在现实设定下(未言明约束必须自己发现),最好配置平均分只有 0.56,95 个任务里仅 14 个拿到 0.8 以上且无事故。\n\n两个发现值得展开:\n\n- **发现未言明约束是首要瓶颈**:论文作者自述,把这些约束藏起来,头部模型平均掉 0.28 分;而误导性的需求描述,代价只是再多掉一点。\n- **知道不等于做到**:在最好配置成功从干系人处拿到全部关键事实的 23 个任务里,也只有 10 个达到 0.8 以上的业务提升。信息收集解释不了全部差距。\n\n更扎心的是,弱配置经常把客户系统改得比不动还糟。\n\n## 所以呢\n\n这份基准给当下的 agent 榜单热泼了盆冷水:SWE-bench 类「任务卡式」评测的高分,与真实客户环境里的交付能力之间,隔着一整条鸿沟。对采购方,FDE-Gym 式的评测——按客户 KPI 打分、沟通有成本、事故一票否决——比榜单排名更接近该问的问题。对个人开发者,项目方还开了在线练习平台(11 月 1 日对人类用户开放),把 FDE 技能做成了可刷题的训练场:写代码交给 agent 之后,「问对人、找对真问题、为线上结果负责」反而成了人的护城河。\n\n参考:arXiv:2610.22350 (https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.22350)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.22350","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"6ad31a14-c0da-42df-81fd-564281f768db","agentic-ai",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"120fa59a-ff6f-4537-9bf5-f818df636a0e","benchmark",{"id":19,"name":20,"slug":20,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":22,"name":23,"slug":23,"description":14,"color":14},"b9bd9039-fcdb-41a8-b85b-fc1587def2b9","open-source",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"80bb5db1-1614-41fa-8d0c-15a3edf0d3f1","en","FDE-Gym: Best Agent Passes Only 14 of 95 Delivery Tasks","FDE-Gym tests AI agents on 95 real delivery tasks across 18 sectors, scored by customer KPI. Best of 13 configs: mean 0.56, only 14 tasks resolved.","Existing agent benchmarks share one premise: the task is well-specified — an issue report, a spec, a ticket. Real customer delivery is nothing like that. Requirements are ambiguous, key constraints are never written down, and the system is live. FDE-Gym, submitted to arXiv on October 15, targets exactly this gap and quickly climbed the Hugging Face Daily Papers board with 285 upvotes.\n\n## How the benchmark works\n\nBuilt by a 26-author team at DeepLethe, FDE-Gym packs 95 end-to-end delivery tasks drawn from 20 customer scenarios across 18 sectors, each adapted by industry experts from real delivery projects spanning banking, healthcare, logistics, manufacturing and energy. The agent is placed in the role of a forward-deployed engineer: inside a simulated customer environment, it interacts with scripted stakeholders — where messages, trial runs and waiting time all carry penalties — and modifies a production-like system until it delivers a working solution.\n\nScoring ignores what the agent did and measures how much the customer's own number improved: normalized between 0 (system untouched) and 1 (expert reference solution). The GitHub repository (open source, Apache 2.0) phrases it more bluntly: on a 100-point scale, 80 or more counts as accepted, and an incident in production caps the score at zero.\n\n## Results: information is not the bottleneck\n\nThe team evaluated 13 model configurations from seven providers. In the realistic setting, where unstated constraints must be discovered, the best configuration achieves a mean score of just 0.56 and cleanly resolves only 14 of the 95 tasks.\n\nTwo findings deserve attention:\n\n- **Discovering unstated constraints is the primary bottleneck.** Per the authors, withholding those constraints costs leading models 0.28 on average; a misdirected request costs little more.\n- **Knowing is not doing.** On the 23 tasks where the best configuration obtained every key fact from stakeholders, it still reached a business uplift of 0.8 or above on only 10 of them.\n\nWorse, weaker configurations often leave the customer worse off than doing nothing at all.\n\n## So what\n\nThis benchmark pours cold water on the current leaderboard race: high scores on ticket-style evaluations like SWE-bench say little about delivering in a live customer environment. For buyers, the questions FDE-Gym forces — score by the customer's KPI, charge for every interaction, one incident kills the run — are closer to what procurement should ask than any benchmark rank. For individual developers, the team is opening an online practice platform for humans on November 1, 2026, turning FDE skills into a trainable gym: once the agent writes the code, asking the right person, finding the real problem, and answering for what happens live become the human moat.\n\nReference: arXiv:2610.22350 (https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.22350)","fde-gym-agent-delivery-benchmark","2026-10-18T23:08:06Z","2026-10-18T23:08:12.299596Z","2026-10-18T23:08:12.299609Z",true,"agent",157,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"aa718f3c-14cf-4984-8762-794eaaee8c16","MAI-Gym:9B 智能体在 2 万个并行环境里做在线 RL","mai-gym-gui-agent-online-rl-9b","2026-10-21T09:30:00+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"206ea36a-1eea-463f-a241-1e3b32f5ec2d","USTC GraphForge:证据图把任务和 rubric 钉在一起,Qwen3.6-27B 涨三基准","graphforge-ustc-qwen36-27b-evidence-graph","2026-10-04T03:05:00+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"4977d1f4-c8f7-480c-aaa1-ec01d69f44d8","8B 拿 SFT+GRPO 打 685B MoE:KaliBench 把 LLM 网络安全工具调用拆到命令行级","kalibench-cybersecurity-cli-runtime-rewards","2026-10-03T05:00:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"9ebb888c-dfe7-416a-9940-a913527d4f73","AI Agent 的失败比成功更值钱:5 万对错误诊断数据,修正通过率 18.4%→51.1%","agent-error-dataset","2026-10-01T15:11:08+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"e98cf9a2-8348-4142-9a56-c11166774798","SpeakerMem-R1:多方对话记忆,分清谁说了什么","speakermem-r1-multi-party-memory","2026-09-24T19:05:00+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"089195fb-7fe5-4ba9-a4bc-8e356fe5e923","BAAI把1000个GitHub仓库蒸馏成5000个技能,科研agent奖牌率31%冲到73%","baai-disco-repo-to-skill-library","2026-09-03T17:07:35+00:00"]