[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-gdp-xlsx-excel-frontier-model-benchmark":3,"topics-all":38,"news-related-b6285867-8ac1-4e91-bc05-d9f4a9ee7210":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"b6285867-8ac1-4e91-bc05-d9f4a9ee7210","20个前沿模型做Excel,榜首只过38.3%","Surge AI 发布 GDP.xlsx:13 个行业从业者写的 70 条真实请求,配 218 万个单元格的工作簿,AI 评审按 791 条专家标准逐条判分,全对才算过。20 个模型配置里,榜首 Gemini 4 Argon 通过率 38.3%,中位数 19.3%,失败多在漏答集合成员、漏看批注筛选与贴图里的证据。","各家模型的基准成绩单越刷越漂亮,可把它们放到一张同事留下的真实工作簿面前,画面就不一样了。Surge AI 团队 9 月 30 日上线了 GDP.xlsx 榜单:70 条由行业从业者撰写的真实职场请求,来自 12 家开发商的 20 个模型配置同场作答,按「全对才算通过」的严格口径,没有一个通过率超过四成。论文 11 月 13 日挂上 arXiv,已被 Agent Evaluation Science Symposium(11 月在纽约举办)接收。\n\n## 这份考卷有多真\n\nGDP.xlsx 的 70 条请求全部出自 13 个领域的从业者之手,起点是他们工作流里前沿模型真的翻过车的场景,每条任务再过三轮评审。输入文件是 58 个 Excel 工作簿加 12 个 CSV,合计 218 万个非空单元格;请求本身极短,中位数只有 55.5 个词,而且 70 条里有 58 条不会告诉你「去看哪个 tab」。论文作者是 Surge AI 的 Suhaas Garre、Sushant Mehta 与 Edwin Chen。\n\n## 考法:全对才算过\n\nAgent 在容器里作答:有终端和文件编辑器,Python 带 openpyxl、pandas,还有 LibreOffice 和 Tesseract OCR;一次尝试最多 200 轮、60 分钟,容器内只有模型 API 可达,证据必须来自文件本身。评分端不是字符串比对,而是一个能打开文件、跑代码验证的 OpenCode 评审 agent,按每题 3 到 30 条、平均约 11 条专家标准逐条裁决,先写理由再给结论。strict pass@1 要求一次尝试命中全部标准,39 道题同时要求集合完备与答案排他——多报一项,也算失败。\n\n## 榜单:头部也难看\n\n按论文与官方榜单:Gemini 4 Argon 以 38.3% 排在榜首(任务抽样标准误至多 5.8 个点),Claude Opus 5.5 为 30.3%,Claude Sonnet 5.5 为 29.1%;此外没有任何配置超过 25%,20 个配置的中位数只有 19.3%,恰好是榜首的一半。中游相当拥挤:GPT 6 Astra 22.9%、Muse Spark 1.3 22.6%、Grok 4.7 22.3%、GLM 5.3 21.4%、Qwen 3.8 Max 20.3%、Kimi K3 13.4%,Mistral Large 3 则是 0.0。迭代是有效的:Gemini 4 Argon 比 Gemini 3.1 Pro 高出 28 个点;榜单上同模型的新旧两版,也都是新版得分更高。\n\n## 输在哪:证据不在单元格里\n\n失败案例的共同点,是答案漏掉了文件里确实存在的证据。Muse Spark 1.3 做租金预测,工作簿有 15 个 tab、15910 个非空单元格、9857 条公式,它召回 9 条相关租约中的 6 条、精确率 6\u002F6,依然算失败——漏掉的 3 条只记录在 BreakClauses 页,其中一条还被 AutoFilter 藏了起来,而这 3 条对应着 1400 万美元量级的三年预测净租金。GPT 6 Astra 做临床数据字典,8 个 housebound 代码只交出 3 个;做认证跟踪题时,它声称工作簿没写明谁负责待定认证,而另一个 tab 恰恰记录着这件事。Gemini 3.8 Flash 找设备机柜,漏掉了只标注在贴入 Sheet1 的示意图里的 +MDY50UC017。\n\n结构本身就是数据:58 个工作簿里,42 个用单元格填充色表意,34 个用非默认字体颜色,21 个嵌入了图片或形状,17 个藏着隐藏的表、行或列,中位数的工作簿带 4 项这类特征。论文点破的盲区是:把表格加载成 dataframe,格式、批注、绘图会全部丢失——而证据恰恰常在那里。\n\n## 所以呢\n\n论文的失败分析给了一个朴素的启示:几行 Python 就能列出工作簿里的全部 tab、批注、筛选和嵌入图片,四个失败案例全都败在「没先做检索」。想拿 AI 处理业务表格的人,这篇论文等于一份冷水说明书:榜首也做不到五次尝试里两次全对。基准自身也做了防污染:3 道公开任务配上 Apache 2.0 协议的评测 harness,另外 67 道 held out,还嵌了 canary 串,要求别进训练集。模型的推理在变强,可「把一张表读完整」眼下仍是人的活——你敢把它先发给你的账期表试试吗?\n\n(参考:论文 https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.17766 ;评测 harness https:\u002F\u002Fgithub.com\u002Fsurge-ai\u002Fgdp-xlsx ;在线榜单 https:\u002F\u002Fsurgehq.ai\u002Fbenchmarks\u002Fgdp-xlsx )","https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.17766","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"6ad31a14-c0da-42df-81fd-564281f768db","agentic-ai",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"5e628969-6d2a-437f-998a-104e4b16cfb1","ai-progress",{"id":19,"name":20,"slug":20,"description":14,"color":14},"120fa59a-ff6f-4537-9bf5-f818df636a0e","benchmark",{"id":22,"name":23,"slug":23,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"4bc764ff-e809-4c52-af55-5fcbfaa5a077","en","AI vs the Office Workbook: Top Model Passes Only 38.3%","GDP.xlsx: 70 workbook tasks, 20 frontier configs, strict grading. Top 38.3%, median 19.3%. Misses: set members; evidence in comments and images.","Benchmark scores keep climbing, yet hand a frontier model a workbook inherited from a colleague and the picture changes fast. On September 30 the Surge AI team launched GDP.xlsx: 70 real workplace requests written by professionals, 20 configurations from 12 developers competing, and under all-criteria grading not one passes 40% of attempts. The paper, accepted to the Agent Evaluation Science Symposium in New York, hit arXiv on November 13.\n\n## How real the test is\n\nAll 70 requests were written by professionals across 13 domains, each starting from a workflow where frontier models had failed them, and every task passed three rounds of review. Inputs are 58 Excel workbooks plus 12 CSV files, about 2.18 million populated cells in total. Requests are short, with a median of 55.5 words, and 58 of the 70 never tell the agent which tab to check. The authors are Suhaas Garre, Sushant Mehta, and Edwin Chen of Surge AI.\n\n## The rule: every criterion or fail\n\nAgents work in a container with a terminal and a file editor, Python with openpyxl and pandas, plus LibreOffice and Tesseract OCR; each attempt gets up to 200 turns and 60 minutes, and only the model API is reachable, so evidence must come from the file itself. Grading is not string matching: an OpenCode judge agent can open the file and run code, ruling on each of the 3-to-30 criteria per task, about 11 on average, and writing its reasoning before each verdict. Strict pass@1 demands every criterion; 39 tasks also pair completeness with exclusivity, so one extra item is a fail.\n\n## The leaderboard: ugly at the top\n\nPer the paper and the official leaderboard: Gemini 4 Argon leads at 38.3% (task-sampling standard error of at most 5.8 points), Claude Opus 5.5 at 30.3%, Claude Sonnet 5.5 at 29.1%; no other configuration exceeds 25%, and the median across the 20 configurations is 19.3%, half the leader. The middle is crowded: GPT 6 Astra 22.9%, Muse Spark 1.3 22.6%, Grok 4.7 22.3%, GLM 5.3 21.4%, Qwen 3.8 Max 20.3%, Kimi K3 13.4%, and Mistral Large 3 at 0.0. Iteration works: Gemini 4 Argon sits 28 points above Gemini 3.1 Pro, and wherever the leaderboard holds two versions of a model, the newer scores higher.\n\n## Where it fails: evidence outside the cells\n\nThe shared thread in the failure analysis is answers that omit evidence the file actually holds. Muse Spark 1.3, on a rent-forecast task over a workbook with 15 tabs, 15,910 populated cells and 9,857 formulas, recalled 6 of 9 tenancies with precision 6 of 6, and still failed: the three it missed live on a BreakClauses tab, one hidden by an AutoFilter, and they carry roughly $14M of three-year forecast net rent. GPT 6 Astra delivered 3 of 8 housebound codes in a clinical data dictionary, and on a certification tracker claimed the workbook never says who handles a pending certification, while another tab records exactly that. Gemini 3.8 Flash missed a cabinet labeled only in a diagram pasted onto Sheet1, the +MDY50UC017 unit.\n\nStructure is data: of the 58 workbooks, 42 use cell fill colors to carry meaning, 34 use non-default font colors, 21 embed images or shapes, and 17 hide sheets, rows, or columns; the median workbook carries four such features. The blind spot the paper names: loading a workbook into data frames keeps its values and drops formats, comments, and drawings, which is precisely where the evidence often sits.\n\n## So what\n\nThe failure analysis offers a plain lesson: a few lines of Python list every tab, comment, filter, and embedded image in a workbook, and all four failures come down to skipping that retrieval step. For anyone pointing AI at business spreadsheets, the paper is a bucket of cold water: even the leader passes fewer than two attempts in five. The benchmark itself guards against contamination, with three public tasks, an Apache 2.0 harness, 67 held-out tasks, and a canary string asking that it stay out of training data. Reasoning keeps improving; reading a workbook to the end is still a human job. Would you send it your receivables file first?\n\nRefs: paper https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.17766 ; harness https:\u002F\u002Fgithub.com\u002Fsurge-ai\u002Fgdp-xlsx ; leaderboard https:\u002F\u002Fsurgehq.ai\u002Fbenchmarks\u002Fgdp-xlsx","gdp-xlsx-excel-frontier-model-benchmark","2026-11-16T21:40:00Z","2026-11-16T21:11:52.217706Z","2026-11-16T21:11:52.217714Z",true,"agent",115,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"6c8ae96a-53e0-4ef7-8609-40b345cdee7c","决策模型实测:自己演砸42.9%,教学生82.9%","decision-model-actor-judge-teacher","2026-11-15T21:45:00+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"bd350599-7253-4f32-a874-9224bab89a49","多智能体搜索实测：32组对照里委派多数时候反而更差","multi-agent-search-delegation-accuracy","2026-11-15T09:10:00+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"a2038f05-8ad2-4b57-bb29-47ff522e09a0","失败轨迹偷走权重:ALFWorld成功率涨7到9.4点","outcome-mass-alignment-multi-turn-agent-rl","2026-11-14T11:20:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"9531b4f7-3f4d-474c-b233-c715925a3e76","8.4B轨迹压过25.3B技能语料:形式化证明中训配方","davinci-prover-agent-native-mid-training","2026-11-14T07:20:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"343f6cae-ae14-4617-a317-7316748e1d6e","删掉推理链再转发:同档32B模型差51分","deletion-sensitivity-reasoning-trace-routing","2026-11-13T07:05:00+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"365a0af2-d564-4eff-bf03-a138cdea0c5f","研究智能体大考:只查交付过69%,查全要求剩28%","are-bench-research-agent-context-requirements","2026-11-13T01:00:00+00:00"]