[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-saturn-ai-finance-benchmark-57-percent":3,"topics-all":38,"news-related-96ebc302-b442-44fe-a23d-d08befbc1246":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"96ebc302-b442-44fe-a23d-d08befbc1246","Saturn 报告:18 款主流 AI 答理财错 57%","Saturn《Artificial Authority》报告实测 18 款 AI 答 121 道理财题:平均错 57%,复杂题错 88%,部分模型 99%。最强 Claude Opus 5 推理模式仍有 39% 错误,失误多为计算错、漏税改、凭空编规则。","英国金融科技公司 Saturn 在 9 月发布的《Artificial Authority》报告,把 18 款主流 AI 模型按真实理财问题轮番考了一遍。结论并不好看:平均 57% 的回答存在错误,涉及多步计算的复杂题,错误率直接跳到 88%;部分模型在最难的题目上错率高达 99%。表现最好的 Claude Opus 5 推理模式,正确率也只到 61%——也就是说,即便你选了榜单第一,每三次提问里仍会有一次拿到错的答案。\n\n## 测试长什么样\n\n这次测评覆盖 ChatGPT、Gemini、Claude、Copilot、Grok 五家厂商的免费与付费版本,共 18 个模型,问题库 121 道,每道题最多重复 5 次,累计 10,000 余次问答。题目按难度分三档:简单(术语解释类)、中等(单一计算)、复杂(多步计算 + 税法交互)。评分采用「paraplanner 必须通过清单」+ LLM-as-judge 的方式,只有模型回答覆盖全部要点才算通过,否则记为失败。\n\n报告里点了三类高频失误:计算错误(尤其是税率、养老金边界)、遗漏即将生效的税收政策变更、凭空编造不存在的规则。FT 的报道(https:\u002F\u002Farchive.is\u002FWD6Vy,原网址 https:\u002F\u002Fwww.ft.com\u002Fcontent\u002Fc0cd359d-df84-4208-a789-ffa864b43666)援引了两个具体例子:Claude Haiku 4.5 在一道养老金税务问题上出错,差点让用户被 HMRC 罚 17,500 英镑;另一例中 Claude「发明」了一条规则,声称毕业生只要搬到国外就可以停还学生贷款。\n\n## 监管与业界的反应\n\nSaturn CEO Amal Jolly 在 FT 文中直言,「数以百万计的人正用 AI 模型问理财问题,但他们拿到的答案会让他们亏钱」。报告发布同时撞上了英国金融市场行为监管局(FCA)的另一组数据:五分之一的英国成年人愿意让 AI 替自己做财务决策,越年轻越愿意。Saturn CEO 借此呼吁 FCA 加快把 AI 理财建议纳入监管,因为目前这类服务不受监管,消费者既没有赔偿渠道,也没有追责对象。AJ Bell 个人理财主管 Sarah Coles 的回应更克制:AI 仍然适合做预算和初步研究,但凡涉及税务、债务优先级、退休规划,「专业顾问的价值就在这里,不能交给 AI」。\n\n## 方法学争议\n\n不过这份报告自己也挨了不少板砖。最尖锐的批评集中在方法学:评分用的是 LLM-as-judge,而它要测评的对象恰恰是 LLM——这等于用选手当裁判;评分标准里「漏掉风险提示」「漏掉合规步骤」这类不完全性失误占比不低,有人算过大约 37% 的失败案例并非事实错,而是「不全」;Saturn 本职是向 750 家理财顾问公司卖 AI 合规工具,利益相关明显。报告本身需要填邮箱才能下载,题目清单与具体 prompt 也未公开,第三方很难复现。FT 文章评论区里有人说得很直接:「这家公司卖的就是 AI 解决不了合规问题,所以才更需要人,那他们做的研究当然要得出 AI 不行的结论。」\n\n## 这事到底意味着什么\n\n把这些争议摆一边,数字本身仍然值得关注。88% 的复杂题错误率不是营销话术——它意味着,在多步计算 + 跨规则联动的场景里,主流大模型的理财问答能力远低于普通用户的直觉预期。再加上 LLM 的「自信口吻」与「礼貌错误」特性,用户在没有专业知识校验的情况下,极难自己识别错误。\n\n实操层面有几条简单规则可以套用:涉及金额、税号、边界条件的提问,务必让模型分步列出算式并标注假设;对涉及「即将生效」政策的提问,直接对照政府官网原文;把 AI 当成「初稿生成器 + 对照表」,而不是「顾问」。当模型说「你可以……」的时候,默认它错了,再去证真。\n\nSaturn 这份报告最大的价值,不是把 AI 钉在耻辱柱上,而是给「AI 替代专业服务」的热潮降了一档温:在专业门槛高、责任链条长的领域,「平均能用」还不够,得「基本不出错」才敢真用。下次再有 AI 理财工具拍胸脯,先让它跑一遍这 121 道理财题,再看它怎么吹。","https:\u002F\u002Farchive.is\u002FWD6Vy","8fc22e42-e2dd-442b-84e4-77eb68061c39",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"23544f6a-eea1-4f05-aa8d-749ca862d5d2","anthropic",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"120fa59a-ff6f-4537-9bf5-f818df636a0e","benchmark",{"id":19,"name":20,"slug":20,"description":14,"color":14},"dca4d0ab-7994-43a7-839e-7756fc77344a","claude",{"id":22,"name":23,"slug":23,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"581a7102-4d90-439a-8917-5ef3b4debcb5","en","Saturn: 18 AIs answer finance questions, 57% wrong","Saturn tested 18 AI models on 121 finance questions: 57% wrong on average, 88% on hard, 99% on the hardest. Best, Claude Opus 5 reasoning, 39% wrong.","UK fintech firm Saturn released its September \"Artificial Authority\" report, putting 18 mainstream AI models through a gauntlet of real personal-finance questions. The verdict isn't flattering: 57% of answers contained errors on average. On multi-step, calculation-heavy questions, the error rate jumped to 88%; the hardest items saw some models fail 99% of the time. Even the best performer — Anthropic's Claude Opus 5 in reasoning mode — landed at only 61% accuracy. Pick the top of the leaderboard and you still get a wrong answer roughly one in three times.\n\n## What the test looked like\n\nThe benchmark covers free and paid tiers from ChatGPT, Gemini, Claude, Copilot and Grok — 18 models in total — against 121 money-related questions, repeated up to five times each, for over 10,000 answers in aggregate. Questions fall into three difficulty buckets: easy (definitions), medium (single calculation), and hard (multi-step computation combined with tax-rule interactions). Grading uses a \"paraplanner must-pass checklist\" combined with an LLM-as-judge; a response only counts as a pass if it hits every required point, otherwise it is marked a failure.\n\nThree failure modes dominate the report: arithmetic errors (especially around tax rates and pension thresholds), omissions of upcoming tax-policy changes, and rules that were simply invented out of thin air. The FT's coverage (https:\u002F\u002Farchive.is\u002FWD6Vy, original URL https:\u002F\u002Fwww.ft.com\u002Fcontent\u002Fc0cd359d-df84-4208-a789-ffa864b43666) cites two concrete examples. Claude Haiku 4.5 miscalculated a pension-tax scenario that could have exposed the saver to a £17,500 HMRC charge. In a separate case, Claude \"invented\" a rule claiming a UK graduate could stop student-loan repayments simply by moving abroad.\n\n## Reactions from regulators and industry\n\nSaturn CEO Amal Jolly told the FT plainly that \"millions of people are trusting AI models for money advice, but they are getting wrong answers that can lose them money.\" The report dropped alongside a UK Financial Conduct Authority finding that one in five UK adults is open to letting AI make financial decisions for them, with the willingness skewing younger. Jolly used the moment to push the FCA to bring AI financial advice inside the regulatory perimeter — currently it sits outside, so consumers have neither compensation rights nor a clear path to redress. Sarah Coles, head of personal finance at AJ Bell, gave a more measured response: AI is fine for budgeting and initial research, but anything involving tax, debt prioritisation, or retirement planning \"is exactly why professional advice still matters.\"\n\n## Methodology pushback\n\nThe report has not gone unchallenged. The sharpest critique hits the methodology: the grader is itself an LLM, judging the same kind of object it is supposed to be judging — referee and player are the same team. A large slice of the \"failures\" (~37% by one commenter count) are flagged for incompleteness rather than factual error — missing a required warning, missing a compliance step — which is a softer failure mode than the headline implies. Saturn's day job is selling AI compliance tools to 750 financial-advisory firms, so the commercial angle is visible. The full report sits behind an email gate, the question list and prompts aren't public, and third parties can't reproduce the run. One FT commenter cut straight to the point: \"A company selling the answer to the problem publishes research concluding the problem is real — go figure.\"\n\n## What this actually means\n\nSetting the methodology debate aside, the headline numbers still deserve attention. An 88% error rate on complex items is not marketing copy; it means that in multi-step, cross-rule scenarios, the financial-QA capability of mainstream LLMs is well below what a casual user would assume. Combine that with the model's confident tone and its polite, fluent delivery of errors, and a non-expert has almost no way to spot a wrong answer on their own.\n\nA few practical rules for anyone using AI for money questions: when the prompt touches numbers, thresholds or jurisdiction, force the model to show the calculation step by step and to label every assumption. For any question about a policy that \"takes effect soon,\" go to the primary government source and compare. Treat the model as a draft generator and a sanity-check table — not as an adviser. When the model says \"you can…\", assume it's wrong until you've proven otherwise.\n\nSaturn's biggest contribution is not a public shaming of AI. It's a temperature check on the \"AI replaces the professional\" narrative. In domains with high expertise floors and long liability tails, \"usually works\" is not enough; \"rarely fails\" is the bar. Next time an AI finance tool pats itself on the back, ask it to run this 121-question battery first — and see how confident it still sounds.","saturn-ai-finance-benchmark-57-percent","2026-09-26T05:04:42Z","2026-09-26T05:05:30.345958Z","2026-09-26T05:05:30.345972Z",true,"agent",608,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"6cd8c9b7-b493-4ad7-bd20-a907b9fb70f4","Claude 5.5 家族首发:Opus 5.5 缓存读取降六成,典型负载省四成","claude-opus-5-5-cost-cut-benchmarks","2026-09-22T21:10:00+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"a7a8ca05-6134-4b27-8dc0-ae3c0a764630","Claude 自主设计量子电路:53 小时拿到一个数量级提速","claude-quantum-circuit-redesign","2026-10-19T10:00:00+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"2c1d70a6-02aa-4b17-9413-a8663216baf6","Anthropic 网络验证改三层:Glasswing 并入","anthropic-glasswing-cyber-verification-tiers","2026-10-15T09:10:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"acddc7d5-c389-4e90-8a7a-1001ce693395","Anthropic 反蒸馏到第三层:误拦也要收费,合法用户背成本","anthropic-distillation-shields-claude-paywall-blocks","2026-10-14T03:00:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"a5c82cbd-def9-4d33-99b3-cba1d7ab4abe","Haiku 5.5 发布:输入一毛钱,跑分最高翻4.6倍","claude-haiku-5-5-launch","2026-10-07T21:09:42+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"090954a9-c27b-46c0-a864-a10fe10a2331","70.6% 的中端模型:Sonnet 5.5 终端评测越级压过 Opus 5.5","claude-sonnet-5-5-terminal-bench","2026-09-28T23:05:32+00:00"]