[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-iclr-2026-multiturn-conversation-39pct-drop":3,"news-related-9c2b49ce-a543-48cb-932d-b36daea3035c":36},{"id":4,"title":5,"summary":6,"content":6,"original_url":7,"source_id":8,"tags":9,"translations":23,"news_slug":29,"published_at":30,"created_at":31,"modified_at":32,"is_published":33,"publish_type":34,"image_url":13,"view_count":35},"9c2b49ce-a543-48cb-932d-b36daea3035c","ICLR 2026杰出论文警示：LLM在多轮对话中平均性能暴跌39%","LLM训练数据与部署场景之间存在一个长期被忽视的鸿沟：训练数据大多是单轮文本补全任务，而实际部署却以多轮对话为主。ICLR 2026杰出论文《LLMs Get Lost In Multi-Turn Conversation》通过大规模模拟实验，系统地揭示了这个问题的严重程度——这或许是目前对LLM实用能力最有力的一次揭短。\n\n核心发现：39%的平均跌幅\n\n研究团队设计了一套可扩展的多轮对话能力评估方法，在六种生成任务上对比了单轮与多轮场景下模型的表现。结果令人警醒：当前头部开源和闭源模型在多轮对话中的表现均显著下滑，平均性能跌幅达39%。\n\n更值得关注的是，研究进一步将性能衰减分解为两个维度：能力的小幅下降，以及可靠性的急剧降低——这意味着模型不仅答得更差，而且答错的频率也大幅增加。换言之，用户面临的不只是质量下滑，而是信任成本的全面上升。\n\n问题根源：过早承诺，无法纠偏\n\n研究分析了超过20万轮模拟对话后发现，LLM在早期对话轮次中过早做出假设并急于生成最终答案，随后过度依赖这些早期假设——一旦方向错误就无法重新纠偏。用研究者的话说：当LLM在对话中走错了一步，它们就会迷路，再也回不来。\n\n这个问题与当前模型评估体系的核心偏差直接相关：几乎所有主流模型的能力评估都以单轮、完全指定的任务为核心，但用户的真实使用场景往往需要多轮澄清、逐步明确需求。两个场景之间存在系统性偏差，模型的Benchmark成绩无法真实反映用户在多轮交互中的体验。\n\n对行业的警示\n\n这项研究的意义不仅在于发现问题，更在于提供了一个可扩展的诊断方法——让多轮对话能力的评估成为可能而非主观感受。对模型开发者而言，这意味着需要重新思考训练数据的构成，引入更多多轮对话样本，而非全部依赖单轮补全数据；对评估框架而言，单轮Benchmark独大的现状需要被打破，多轮场景应该拥有独立的评估track；对应用开发者而言，在关键场景中需要设计更robust的人机协作机制，在模型迷路时给用户足够的修正机会，而不是放任它一路错下去。\n\n仅靠扩大参数规模和单轮benchmark成绩来评估模型，已经不够了。多轮对话能力正在成为LLM从玩具走向工具的关键短板。39%的数字值得所有从业者警醒。","https:\u002F\u002Fopenreview.net\u002Fforum?id=VKGTGGcwl6","ec0a79b7-694c-4caf-8071-91315d69c706",[10,14,17,20],{"id":11,"name":12,"slug":12,"description":13,"color":13},"5e628969-6d2a-437f-998a-104e4b16cfb1","ai-progress",null,{"id":15,"name":16,"slug":16,"description":13,"color":13},"120fa59a-ff6f-4537-9bf5-f818df636a0e","benchmark",{"id":18,"name":19,"slug":19,"description":13,"color":13},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":21,"name":22,"slug":22,"description":13,"color":13},"01598627-1ea6-4b27-a5d8-874971571a71","llm",[24],{"id":25,"lang":26,"title":27,"summary":28,"content":13},"cc77d1b2-bb2f-4639-a17d-0c2fad9895fc","en","ICLR 2026 paper: LLMs drop 39% in multi-turn dialogue","There's a long-overlooked gap between LLM training data and deployment scenarios: training data is mostly single-turn text completion tasks, while actual deployment is dominated by multi-turn conversation. The ICLR 2026 outstanding paper \"LLMs Get Lost In Multi-Turn Conversation,\" through large-scale simulation experiments, systematically reveals the severity of this problem — this may be the strongest takedown of LLM practical capability to date.\n\n**Core finding: 39% average drop**\n\nThe research team designed a scalable evaluation method for multi-turn dialogue capability, comparing model performance in single-turn vs multi-turn scenarios across six generation tasks. The results are sobering: current top open-source and closed-source models show significant performance drops in multi-turn dialogue, with average performance decline of 39%.\n\nMore noteworthy, the research further decomposes performance decay into two dimensions: a slight decline in capability, and a sharp decline in reliability — meaning the model not only answers worse, but also fails more often. In other words, users face not just quality decline, but a comprehensive rise in trust cost.\n\n**Problem root: premature commitment, unable to correct**\n\nAfter analyzing over 200,000 simulated dialogue turns, the researchers found LLMs make premature assumptions in early dialogue turns and rush to generate final answers, then over-rely on these early assumptions — once the direction is wrong, they can't correct. In the researchers' words: when LLMs take a wrong step in conversation, they get lost and can never come back.\n\nThis problem is directly tied to a core bias in the current model evaluation system: nearly all mainstream model capability evaluations are centered on single-turn, fully specified tasks, while users' real use scenarios often need multi-turn clarification and progressive requirement specification. There's a systematic bias between the two scenarios, and model benchmark scores can't truly reflect user experience in multi-turn interaction.\n\n**Warning to the industry**\n\nThe significance of this research goes beyond problem identification — it provides a scalable diagnostic method, making multi-turn conversation capability evaluation possible rather than subjective impression. For model developers, this means reconsidering training data composition, introducing more multi-turn dialogue samples rather than all single-turn completion data; for evaluation frameworks, the dominance of single-turn benchmarks needs to be broken, with multi-turn scenarios deserving their own evaluation track; for application developers, more robust human-machine collaboration mechanisms need to be designed in key scenarios, giving users enough correction opportunities when models get lost, rather than letting them go all the way wrong.\n\nRelying on parameter scaling and single-turn benchmark scores to evaluate models is no longer enough. Multi-turn conversation capability is becoming the key shortcoming for LLMs transitioning from toys to tools. The 39% number deserves every practitioner's attention.","iclr-2026-multiturn-conversation-39pct-drop","2026-05-04T10:20:00Z","2026-05-04T10:26:17.695175Z","2026-08-19T02:08:40.142862Z",true,"agent",138,{"items":37},[38,43,48,53,58,63],{"id":39,"title":40,"news_slug":41,"published_at":42},"e2eb81dc-3112-411a-94b0-b5061a12be78","AdvancedMathBench 把数学证明拉进博士级:GPT-5.5-xhigh 仍只 75.8","advanced-math-bench-phd-level","2026-07-14T16:15:00+00:00",{"id":44,"title":45,"news_slug":46,"published_at":47},"fc1e888d-4bee-4633-86f4-edc76aa48161","BlockSearch 把语言模型变成「上下文检索器」：0.6B 在百万 token 上打平向量检索","blocksearch-context-retriever","2026-07-03T06:25:00+00:00",{"id":49,"title":50,"news_slug":51,"published_at":52},"95b04c15-d5e5-4dca-ab2b-14e343bdd4e6","UC Berkeley 曝光 AI 基准测试系统性漏洞：45 种方法可在 13 个主流榜单上「不解决任何问题拿满分」","uc-berkeley-benchmark-45-cheats-13-leaderboards","2026-05-15T01:00:00+00:00",{"id":54,"title":55,"news_slug":56,"published_at":57},"1311adb6-dc19-41a7-a188-6760d9e53672","HF Summer 2026 报告:13 个下载量 Top 25 模型是 2022 年的老面孔","hugging-face-summer-2026-attention-adoption","2026-08-24T08:00:00+00:00",{"id":59,"title":60,"news_slug":61,"published_at":62},"43eda321-b0b7-4df7-b20e-9758cbab42c9","记忆越完整,眼前题越做不对:MemTrapBench 把 LLM 长期记忆框架打回原形","memtrapbench-llm-memory-cognitive-traps","2026-08-22T04:00:00+00:00",{"id":64,"title":65,"news_slug":66,"published_at":67},"c5304aea-6b80-4fbb-b2ac-f07a7a6d7713","DeepSeek V4 Flash 上午短暂\"翻车\"：国产开源模型的容量大考","deepseek-v4-flash-api-capacity-incident","2026-08-04T10:00:00+00:00"]