[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-neo-classic-poetry-reasoning-benchmark":3,"topics-all":38,"news-related-0138c087-f1c3-44c5-a662-045c6f20a756":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"0138c087-f1c3-44c5-a662-045c6f20a756","ACL 2026 新基准:大模型的古诗美学推理翻车了","ACL 2026 基准 Neo-Classic 用当代新写古诗考模型,Qwen3-Max 等旗舰成绩掉两到五成,排序题仅 0-13%。","古诗词一直是大模型刷分的舒适区。在既有基准上,Qwen3-Max、Gemini-3-Pro 这类旗舰模型对唐诗宋词的判读准确率常年高企。但一篇收录进 ACL 2026 长论文名录的新工作提出了一个尴尬的问题:模型到底是「懂」诗,还是「背」过诗?\n\n研究者搭建了名为 Neo-Classic 的评测基准,专门检验大模型的语言美学推理能力——也就是超越记忆检索、对格律和篇章结构做真正推理的能力。论文公开于 arXiv(编号 2609.19154),并收录于 ACL 2026 会议论文集(第 64 届年会长论文卷,页码 27442-27465)。\n\n## 考卷怎么出:让背书失效\n\n旧基准大多在历史语料上做验证或生成,模型完全可以靠预训练时「见过」来答题。Neo-Classic 换了一条路:基准由一套构造主义的样本外数据集加上一组反向理解探针组成,全部诗歌都由当代专家按严格格律新写,与训练语料天然隔离,直接检索这条路被堵死。配套的五个行为探针层层加码,专门检验层级化的约束满足能力。\n\n## 成绩单:从历史到新诗,集体跳水\n\n研究团队测了 Qwen3-Max、Gemini-3-Pro、DeepSeek-V3.2 三个旗舰模型。结果呈现两个层级的滑坡:第一,同样的题型,把文本从历史语料换成当代新作,性能差距拉出 20% 到 50%;第二,篇章级的排序任务上,标准准确率只有 0 到 13%。给推理增强型模型加上专家级指导后,成绩爬升到 36%,但与人类专家仍有明显差距。\n\n## 局部会背,全局不会想\n\n作者给出的判断很直接:当前大模型能捕捉局部形式模式,但在稳健的语言美学推理所需的全局层级规划上仍然挣扎。换句话说,模型能对上某一联的格律,却难以围绕整首诗的篇章结构做统筹。\n\n## 所以呢\n\n这篇论文值得注意的地方,不在「大模型古诗不行」这个结论本身,而在方法论:用构造主义的样本外数据加反向探针,把「记忆」和「推理」拆开分别计量。对中文能力评测尤其有参考价值——凡是语料里海量存在的文体,高分都可能是背诵的利息。什么时候模型在没人写过的新文本上也能做结构推理,什么时候才算真的读懂。对做评测的人来说,这是一套值得抄的出题思路。\n\n参考:[arXiv:2609.19154](https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.19154) \u002F [ACL 2026 长论文 DOI](https:\u002F\u002Fdoi.org\u002F10.18653\u002Fv1\u002F2026.acl-long.1266)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.19154","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"120fa59a-ff6f-4537-9bf5-f818df636a0e","benchmark",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"a9524a82-a7c5-4daa-bb4b-a7ee77bb0b94","gemini",{"id":19,"name":20,"slug":20,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":22,"name":23,"slug":23,"description":14,"color":14},"c187600e-804c-4697-b828-1e4330e0eb10","qwen",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"4c1cf383-0bf9-4cfc-9a1c-0c9c5a495b74","en","Neo-Classic: Frontier LLMs Slip on New Classical Poetry","ACL 2026 benchmark Neo-Classic tests LLMs on fresh expert-written metrical poetry: Qwen3-Max and peers drop 20-50%, ordering at 0-13%.","Classical Chinese poetry has long been a comfort zone for large language models. On existing benchmarks, flagship models like Qwen3-Max and Gemini-3-Pro score consistently high when judging Tang and Song verse. A new paper in the ACL 2026 long-paper proceedings raises an awkward question: do these models actually understand poetry, or have they merely memorized it?\n\nThe researchers built Neo-Classic, a benchmark designed to evaluate linguistic-aesthetic reasoning — the ability to reason genuinely about meter and discourse structure rather than retrieve from memory. The paper is available as arXiv:2609.19154 and appears in the Proceedings of the 64th Annual Meeting of the ACL (Volume 1: Long Papers, pages 27442-27465).\n\n## How the Test Is Built: Defeating Memorization\n\nMost legacy benchmarks rely on verification or generation over historical corpora, so a model can answer by virtue of having seen the material during pretraining. Neo-Classic takes a different route: it combines a constructionist out-of-sample dataset with a suite of reverse-understanding probes. Every poem in the dataset is strictly metrical verse authored by contemporary experts, which is naturally disjoint from training corpora and blocks the direct-retrieval shortcut. Five behavioral probes layer constraints on top of one another to test hierarchical constraint satisfaction.\n\n## The Scorecard: A Collective Plunge\n\nThe team evaluated three flagship models: Qwen3-Max, Gemini-3-Pro, and DeepSeek-V3.2. Two levels of degradation emerged. First, when the same task switched from historical to contemporary texts, performance dropped by 20 to 50 percent. Second, on discourse-level ordering tasks, standard accuracy ranged from 0 to 13 percent. With expert-level guidance, reasoning-enhanced models climbed to 36 percent — still a notable gap from human experts.\n\n## Local Patterns Yes, Global Planning No\n\nThe authors' verdict is blunt: current LLMs capture local formal patterns but struggle with the global hierarchical planning required for robust linguistic-aesthetic reasoning. In other words, a model can match the meter of a single couplet, yet fail to coordinate the structural arc of an entire poem.\n\n## So What\n\nThe real contribution here is not the headline that LLMs are bad at classical poetry, but the methodology: constructionist out-of-sample data plus reverse probes separate \"memory\" from \"reasoning\" and measure each independently. For Chinese-language evaluation this matters especially — any genre densely represented in the training corpus can earn high scores as interest on memorization. Only when models can reason over texts no one has written before should we call it genuine understanding. For benchmark designers, this is an exam-writing playbook worth copying.\n\nReference: [arXiv:2609.19154](https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.19154) \u002F [ACL 2026 Long Papers DOI](https:\u002F\u002Fdoi.org\u002F10.18653\u002Fv1\u002F2026.acl-long.1266)","neo-classic-poetry-reasoning-benchmark","2026-09-20T15:15:00Z","2026-09-20T15:14:07.295751Z","2026-09-20T15:14:07.295759Z",true,"agent",6,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"47f12a0d-8559-473f-97be-dc12966bd4ff","DeepMind 双盲评测：Gemini 权重和考题锁进同一个加密飞地","deepmind-gemini-double-blind-eval","2026-08-29T15:05:00+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"dcb1506b-87fa-422f-89e0-bb62afcc2b4c","BenchLM 8 月榜:Qwen3.8 Max 79.2 分领跑开源 LLM,MiniMax M3 跻身三强","qwen3-8-max-benchlm-aug-2026","2026-08-28T06:00:00+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"d74a088d-e7f5-41cc-8c55-aedb10b8101d","Gemini-3-Pro 也只拿 66.4 分:南京大学开源全模态视频助手基准 OmniAssistBench","omniassistbench-omni-llm-video-assistant","2026-08-21T17:59:52+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"4c9f74d4-0252-4e86-8b6e-85d38788eea6","开源编程模型三选一:GLM-5.2、DeepSeek V4、Qwen3.6","glm-5-2-deepseek-v4-qwen-3-6-coding","2026-07-27T06:00:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"abccd085-d910-4846-aae1-5f2136238f3b","别怪大模型:Coding Agent 真正的瓶颈在「脚手架」——Queen's 35 版本控制变量实证","coding-agent-scaffold-bottleneck","2026-07-04T03:55:25+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"4455b8ee-eab9-463b-9934-f1df4b1b4fb3","扩散语言模型的适配断点被接上:dQwen3.5 只花一半 token","dqwen3-5-hybrid-attention-diffusion-language-models","2026-09-18T19:20:00+00:00"]