[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-surge-dayjob-tax-agent-benchmark":3,"topics-all":38,"news-related-2a16565e-87a9-4f83-955b-557b9b6ba5bb":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"2a16565e-87a9-4f83-955b-557b9b6ba5bb","AI 税务基准:10.8% 通过率,专业活还是干不动","Surge AI 发布 DAYJOB: Tax 基准:50 个税务专家设计的真实任务,每个任务配一套客户档案,共 1,789 个文件,按满足 95% 评分标准判定通过。Sonnet 5.5 以 10.8% 居首,Fable 5.1 与 Opus 5.5 为 9.6% 和 9.2%,Kimi K3 等三家为 0。","一个税务智能体把两种卖房日期都算对了，却没告诉客户：再拖三个月，最高 25 万美元（$250,000）的免税额度就没了。这不是算错，是漏问。\n\nSurge AI 在 10 月 27 日上线了 DAYJOB: Tax 基准。50 个由执业税务专家设计的专业任务，每个任务配一个装满客户档案的完整工作环境。评测方式是 pass@1：50 个任务，每个任务跑 5 次尝试，一次尝试要满足该任务 95% 以上的评分标准才算过。榜单第一的 Claude Sonnet 5.5 通过率 10.8%，第二名 Claude Fable 5.1 是 9.6%，第三名 Claude Opus 5.5 是 9.2%([原始榜单](https:\u002F\u002Fsurgehq.ai\u002Fbenchmarks\u002Fdayjob-tax))。\n\n## 什么才算\"通过\"\n\n这套评分标准严在两个地方。\n\n一是标准条数多。官方公开的数据卡显示，每个任务的评分细则中位数是 30.5 条，判分覆盖计算过程、支撑证据、后果性问题和最终交付物（[数据集卡片](https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002Fsurgeai\u002FDAYJOB-tax)）。二是分数线高，一次尝试必须命中 95% 以上的细则才算过——也就是说，一个能把大部分事情做对的智能体，只要漏掉一条会改变结论的判断，照样记零分。\n\n任务本身是容器化的 Harbor 格式，跑的时候网络被限制在允许的模型接口上，浏览和装包都被挡掉，所以没法靠现查现抄蒙混过关。50 个任务只公开了 2 个样例，完整的 50 个要邮件申请。\n\n## 分数单\n\nSurge AI 自己跑的榜单，从上到下：Claude Sonnet 5.5 10.8%、Claude Fable 5.1 9.6%、Claude Opus 5.5 9.2%、Muse Wonder 1.0 6%、Grok 4.7 5%、GPT-6 Astra 4.8%、GPT-6.1 Sol 4%、Muse Spark 1.3 3.6%、Gemini 4 Argon 3.2%、Hy4 Preview 1.6%、DeepSeek V4.1 Flash 1.2%、Gemini 3.8 Flash 0.8%、GLM 5.3 0.4%，Kimi K3、Mistral Large 4、Qwen 3.8 Max 三家是 0。\n\n这是评测方自己跑的自家题目，别的机构目前还没有复现版本。但一个规律很难忽略：专业工作一旦需要主动判断，模型之间的差距不是缩小，而是整个梯队一起掉到 10% 以下。\n\n## 错在判断，不在算术\n\nSurge AI 在博客里给了两个案例（[原文](https:\u002F\u002Fsurgehq.ai\u002Fblog\u002Fdayjob-tax)）。\n\n一个是 Anna 卖公寓。她 2024 年 8 月搬走，想比较 2026 年 10 月和 2027 年 1 月哪个成交更划算。智能体正确判断两个日期都满足\"五年内住满两年\"的居住要求，但漏掉了另一半：一旦拖过 2027 年 8 月底，她就不再满足条件。五次尝试里 Astra 一次都没发现这个截止日，Opus 五次全中。\n\n另一个是分居夫妻分割 Roth IRA 和 401(k)。Opus 5.5 五次都基于\"两人法律上未结婚\"给建议，让他们各留各的账户再加一笔现金找平。但这对夫妻在德州共同生活过、在科罗拉多办过仪式，德州承认非正式婚姻，在佛州另行成立的有效婚姻通常仍然有效。事实可能因此反转税负。Astra 在五次里有四次先去确认婚姻状态。\n\n这两个案例的共同点很扎眼：算术部分基本没问题，塌掉的是\"客户没问、但你应该主动去查\"的那一环。\n\n## 难在文件堆\n\n50 个任务共用 1,789 个文件，中位数每任务 30 个，最多 123 个，其中 1,487 个是 PDF、117 个 Excel、38 个 Word，其余是邮件、图片、CSV 和 JSON。里面有意为之的不规范材料，比如一张转售商品的手写收据照片，要读出品名金额再和支出记录对账；比如一家只申报在阿拉巴马州的工业筒仓服务公司，调度记录显示 crews 在五个州干活。\n\n这些不是\"难文件\"，是真实工作里本来就长那样的东西。\n\n## 和旧税务基准差在哪\n\n已有的税务评测各有侧重：TaxBench 问 114 个有明确答案的税务问题，TaxCalcBench 测的是报税数字算得准不准，TaxLab 测智能体能不能主动向模拟纳税人追问信息并完成申报。DAYJOB 的野心更大一步——它要问的是，客户问错了问题时，智能体能不能把活接下来。\n\n## 我的判断\n\n基准的价值不在分数高低，在于它把\"能力\"的定义往前推了一格。以前我们测智能体，是测它会不会做一件事；现在有人开始测它能不能把一件没人说得清楚的事从头负责到尾。这个转向比 10.8% 这个数字重要得多。\n\n但也要泼一盆冷水。榜单是评测方自己跑的，公开样本只有 2 个，完整任务要申请才能拿到，第三方目前无法独立复现。10.8% 这个数该被当成一个方向性信号，而不是终审判决。而且这套基准测的是\"税务\"这一个职业的一个切面，把它直接外推成\"智能体还干不了专业工作\"是过度解读。\n\n真正值得盯住的是那 30.5 条细则背后的东西：模型已经能把给定的题算对，接下来要练的是发现题本身缺了一块。这件事靠继续堆预训练语料解决不了，只能靠带真实后果的反馈信号去训。而带真实后果的反馈，恰好是专业领域最稀缺的那种数据。","https:\u002F\u002Fsurgehq.ai\u002Fblog\u002Fdayjob-tax","8473f53d-ff32-4f31-a110-43c160647d10",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"6ad31a14-c0da-42df-81fd-564281f768db","agentic-ai",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"120fa59a-ff6f-4537-9bf5-f818df636a0e","benchmark",{"id":19,"name":20,"slug":20,"description":14,"color":14},"dca4d0ab-7994-43a7-839e-7756fc77344a","claude",{"id":22,"name":23,"slug":23,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"a57c443f-7e7c-4084-8ce8-5f1cb74adf59","en","Tax Agents Fail 10.8% of Real Tax Jobs","DAYJOB: Tax benchmark: 50 expert tax assignments, 1,789 files. Top agent Claude Sonnet 5.5 passes just 10.8%. Failures are judgment, not arithmetic.","A tax agent worked out both closing dates correctly and still failed to tell the client something: if the sale slipped by another three months, the exclusion worth up to $250,000 would vanish. That is not a calculation error. It is a missed question.\n\nSurge AI put up DAYJOB: Tax on October 27. The benchmark contains 50 professional assignments written by practicing tax experts, each packaged with a full working environment full of client records. Scoring is pass@1: 50 tasks, 5 attempts per task, and an attempt only counts as a pass when it satisfies at least 95% of that task's rubric criteria. Claude Sonnet 5.5 leads with a 10.8% pass rate, followed by Claude Fable 5.1 at 9.6% and Claude Opus 5.5 at 9.2% ([full leaderboard](https:\u002F\u002Fsurgehq.ai\u002Fbenchmarks\u002Fdayjob-tax)).\n\n## What counts as a pass\n\nThe bar is hard in two separate ways.\n\nFirst, the rubrics are long. The published dataset card puts the median number of rubric criteria per task at 30.5, and grading covers calculations, supporting evidence, consequential issues, and the finished deliverable ([dataset card](https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002Fsurgeai\u002FDAYJOB-tax)). Second, the threshold is high: a single attempt must hit at least 95% of the criteria to count. An agent that gets nearly everything right still scores zero if it drops one judgment call that would have changed the conclusion.\n\nTasks use the containerized Harbor format, with network access restricted to permitted model API hosts during execution — browsing and package downloads are blocked, so there is no looking things up on the fly. Only 2 of the 50 tasks are published; the full set has to be requested by email.\n\n## The scoreboard\n\nOn Surge AI's own run, top to bottom: Claude Sonnet 5.5 at 10.8%, Claude Fable 5.1 at 9.6%, Claude Opus 5.5 at 9.2%, Muse Wonder 1.0 at 6%, Grok 4.7 at 5%, GPT-6 Astra at 4.8%, GPT-6.1 Sol at 4%, Muse Spark 1.3 at 3.6%, Gemini 4 Argon at 3.2%, Hy4 Preview at 1.6%, DeepSeek V4.1 Flash at 1.2%, Gemini 3.8 Flash at 0.8%, GLM 5.3 at 0.4%, and Kimi K3, Mistral Large 4, and Qwen 3.8 Max all at zero.\n\nThis is the evaluator running its own questions, and no outside lab has reproduced the numbers yet. But one pattern is hard to miss: once professional work requires active judgment, the gap between models does not narrow — the entire tier drops below 10%.\n\n## Judgment, not arithmetic\n\nSurge AI published two case studies in the release post ([original](https:\u002F\u002Fsurgehq.ai\u002Fblog\u002Fdayjob-tax)).\n\nTake the case of Anna, selling a condo. She moved out in August 2024 and wanted to compare closing in October 2026 against January 2027. The agent correctly concluded that either date satisfies the two-of-five-year residence requirement for the capital gains exclusion. What it missed is the other half: once the sale drifts past the end of August 2027, she stops qualifying. Across five attempts, Astra never flagged that deadline. Opus caught it in all five.\n\nThe second is a separating couple dividing a Roth IRA and a 401(k). Opus 5.5 based all five recommendations on the assumption that the couple were not legally married, telling them to keep the accounts separate with a cash payment to even out the balances. But they had lived together in Texas and held a ceremony in Colorado, and Texas recognizes informal marriages under certain conditions, while a valid marriage established elsewhere may remain valid in Florida. That distinction could change the tax consequences. Astra checked marital status first in four of five attempts.\n\nThe pattern across both cases is uncomfortable: the arithmetic is largely fine. What collapses is the part where you go looking for the issue the client never raised.\n\n## The files are the hard part\n\nThe 50 tasks share 1,789 files, with a median of 30 per task and as many as 123, including 1,487 PDFs, 117 Excel workbooks, 38 Word documents, and the rest emails, images, CSVs, and JSON records. There is deliberately non-standardized material in there — a photograph of a handwritten receipt for merchandise purchased for resale, where the agent has to read item descriptions and amounts and reconcile them against spending records; and an industrial silo-services company that its client assumes operates only in Alabama, while the field dispatch log shows crews working across five states.\n\nThese are not hard files. They are what real work actually looks like.\n\n## How this differs from earlier tax benchmarks\n\nExisting tax benchmarks each focus on one slice. TaxBench asks 114 tax questions with defined answers. TaxCalcBench measures whether the numbers on a return are right. TaxLab tests whether an agent can question a simulated taxpayer, gather missing information, and complete a federal individual return. DAYJOB wants something broader: when the client asks the wrong question, can the agent still take the job on?\n\n## My read\n\nThe value of a benchmark is not the score. It is that it pushes the definition of \"capability\" forward by one notch. We used to ask whether an agent could do a task. Now someone is asking whether it can own a job end to end that nobody specified properly. That shift matters far more than the 10.8%.\n\nCold water is also due. The leaderboard is the evaluator's own run, only 2 sample tasks are public, the full set requires a request, and no third party can independently reproduce it today. Treat 10.8% as a directional signal, not a final verdict. And this benchmark covers one slice of one profession — reading it directly as \"agents cannot do professional work\" would be overreach.\n\nThe thing worth watching is what sits behind those 30.5 criteria. Models can already get a well-specified question right. The next skill to learn is spotting that the question is missing a piece. Continued growth in pretraining corpus will not fix that; only feedback carrying real consequences will, and consequence-bearing feedback is exactly the kind of data professional domains have the least of.","surge-dayjob-tax-agent-benchmark","2026-10-29T09:05:00Z","2026-10-29T09:10:32.968266Z","2026-10-29T09:10:32.968275Z",true,"agent",294,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"50fb4155-e244-4a2f-8ee7-5596db52da9f","多智能体编排大考:最强模型也只解决36.95%任务","workflowarena-multi-agent-orchestration","2026-11-10T15:30:00+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"158f2181-ecf3-4e5c-ac5a-16e5c45c9d9a","千篇论文建成AI实验室:前沿模型74%胜过人类作者","scientist-world-1000-research-environments","2026-11-08T21:08:04+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"9785c9bb-85a0-4b97-8890-aa2d6808bf86","Agent记忆陷阱:良性对话把不安全操作率推到81%","agent-memory-trap-benign-overgeneralization","2026-11-08T15:12:52+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"8857d393-a5f7-4b2a-8439-bf87ac324d4f","数学建模基准自曝:12个模型算对却只得49.82%","tb-mathmodel-agentic-verifiable-math-modeling-bench","2026-11-08T03:15:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"37e0486a-225d-4b44-b189-206bd756210c","多轮 Agent 强化学习提升了什么:46.8% 的真相","multi-turn-agentic-rl-budget-dependence","2026-11-06T01:18:33+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"99c1eb7b-9226-4d61-afdb-5d54f920a85c","研究品味实测:AI 智能体的想法匹配只有人类的 18%","ideataste-llm-research-taste-benchmark","2026-11-04T11:13:00+00:00"]