[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-constrained-decoding-llm-reasoning-retest":3,"topics-all":38,"news-related-cf6d775d-8321-4992-ad8c-16bd8e50b232":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"cf6d775d-8321-4992-ad8c-16bd8e50b232","约束解码不必然伤推理:11 个开源模型重测","一份提交到 arXiv 的新研究,用 11 个开源模型重测「结构化输出会让模型变笨」这一流行说法。结论是:在提示词与输出格式完全对齐的对照下,约束解码并不必然损害推理能力,此前的退化更多来自提示词与输出格式不等价。论文把约束拆成格式、有效性与推理三类,并开源实现库 litelines。","工程圈有个默认共识：给模型套上\"必须输出合法 JSON\"这类硬约束，模型就会变笨。\n\n这个判断流传很广，早前也有研究报告在结构化输出场景下观察到准确率下降。10 月 26 日提交到 arXiv 的一篇新论文（[arXiv:2610.35328](https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.35328)）把这个问题拆开重做，得到的结论是：**在提示词和预期输出格式完全对齐的前提下，约束解码并不必然损害推理能力**；此前观察到的\"退化\"，主要来自提示词与输出格式的不等价，而不是解码器层面的掩码本身。\n\n论文作者来自 Télécom Paris（Institut Polytechnique de Paris）与 Nokia Bell Labs，代码以 litelines 开源（[GitHub](https:\u002F\u002Fgithub.com\u002Falonsosilvaallende\u002Flitelines)）。\n\n## 把\"约束\"拆成三类，而不是有或没有\n\n论文的核心是给约束解码建立了一个三层分类：\n\n- **格式约束（FC）**：只管输出结构，比如必须是合法 JSON、字段必须齐全。\n- **有效性约束（VC）**：在格式之上再屏蔽掉语义上非法的答案，比如把不在候选集里的选项直接 mask 掉。\n- **推理约束（RC）**：更进一步，约束推理轨迹本身，比如强制模型先输出\"让我们一步步思考\"再作答。\n\n论文给出的理论结论是：只要满足\"输出约束不排除正确答案\"这一前提（文中的 Assumption 1），解码器层面的掩码就不会拉低性能；而要拿到增益，还需要额外条件。\n\n## 11 个开源模型、四个家族、十一个任务\n\n实验规模不算小：跨四个模型家族共 11 个开源模型，其中 10 个（Qwen3-1.7B、Qwen3-4B、Qwen3-8B、Qwen3.5-2B、Qwen3.5-4B、Qwen3.5-9B、Qwen3.5-27B、Llama3-8B、Gemma2-9B、SmolLM3-3B）参与任务求解，Qwen3-32B 另作两阶段抽取器使用。\n\n数据集覆盖两个难度层级。标准档是 GSM8K 和 Last Letter Concatenation；推理密集档是 AIME 2026 与作者自建的 LastLetter-Extended（把 Last Letter 从 4 个名字扩展到 8 个）。后者与 AIME 2026 都晚于全部被测模型发布，用来降低数据泄漏风险。\n\n解码策略比了三种：JSON（在输出里加一个 reason 字段）、NL-to-Format（先自由推理、再用第二个模型抽格式）和 InWriting（单次自由推理，用触发词切换到约束模式）。\n\n## 数字：把\"退化\"逐条拆开\n\n在 Last Letter 任务上做纯文本答案对比时，加了 FC+VC 之后，被约束解码的版本全面反超未约束版本：Qwen3.5-9B 从 86.8 涨到 89.9，Qwen3.5-2B 从 62.9 到 68.7，Qwen3.5-4B 从 84.1 到 86.9，Qwen3-1.7B 从 60.7 到 66.1。\n\n换成 JSON 格式后差距同样存在，Qwen3.5-9B 是 89.6 到 91.6。\n\n论文进一步拆分收益来源，发现有效性约束比格式约束更稳定，最高带来 17.4% 的增益。原因不玄学：有效性约束提供了关于正确答案的语义信息，而格式约束只是把输出形状掰成要求的格式——模型本来就能照做时，几乎没有收益空间。\n\n推理约束则表现分裂。强制以\"让我们一步步思考\"开头，在 GSM8K 和 AIME 2026 上通常有提升；但 Qwen3.5-4B 在 AIME 2026 上反而掉了 6.6%。论文的结论很克制：推理约束不是普适手段，它依赖任务和模型，还可能误伤其他合法推理路径。\n\n## 解析成功率：约束最实在的那块收益\n\n如果说准确率上的提升还得挑条件，解析成功率就是无条件的。\n\n在 LastLetter-Extended 上，未加约束的 JSON 输出解析成功率，Llama3-8B 是 0，Gemma2-9B 是 0，Qwen3-1.7B 是 0.7，Qwen3.5-9B 是 60.4，Qwen3.5-27B 是 82.4。加上格式约束后，这些数字分别变成 100、100、96.2、99.3、93.8。\n\n换句话说，如果你的工程里有一条\"模型输出必须能被直接解析\"的硬要求，约束解码解决的正是这件事本身。而且它对大模型同样成立，Qwen3.5-27B 也不例外。\n\n## token 成本：约束不总是更贵\n\n直觉上约束解码应该更费 token，但表里的数据不完全支持。GSM8K 上 Qwen3-1.7B 的 JSON 输出，未约束时平均 221.1 个 token，加上 FC+VC 后降到 129.2；Last Letter 上同一模型则是 192.0 降到 73.4。\n\n真正贵的是 NL-to-Format——它需要额外一次大模型调用做抽取，论文在结论里明确说，这会让 token 用量接近翻倍。折中方案是 InWriting：只把约束施加在最终输出上，论文说实验中额外增加的 token 不到 15 个。\n\n## 论文自己承认的边界\n\n作者没有把话说满，限制部分列得很具体：\n\n- 有效性约束主要只在 Last Letter 与 LastLetter-Extended 上验证，化学分子结构约束、内容过滤只做了 IMDb 上的初步实验；\n- 推理约束要落地需要事先知道哪些轨迹是\"有益的\"，而这在真实场景往往拿不到；\n- 实验用的是相对简单的 JSON schema，\"既要长推理又要复杂输出结构\"的任务被明确留作未来工作。\n\n还有个细节挺有意思：这篇论文自己在声明里写了，作者用免费版 ChatGPT 和 Claude 起草和润色了部分稿件，所有研究问题、实验和代码都出自作者本人。\n\n## 怎么看\n\n抛开标题里那个问句，这篇工作对工程实践最直接的启示，其实不在准确率，而在方法论。\n\n**当你怀疑\"加约束掉分了\"，先别急着怪解码器。** 论文的因果消融指向的是提示词与输出格式的不等价——拿\"无约束 + 自然语言提示\"的分数去对比\"有约束 + JSON 提示\"的分数，比较的其实是两套不同的任务设定，差异未必来自约束本身。论文给出的做法是：固定提示词、固定预期输出格式，再做有约束与无约束的对照。\n\n另一条：约束解码最大的确定性收益在工程侧，不在分数上。把解析成功率从 0 拉到 100，对一条下游流水线意味着少一整层容错代码。\n\n实现也值得看一眼。litelines 是论文配套的实现，`pip install litelines` 即可安装，唯一依赖是 outlines-core，通过 Hugging Face transformers 接入，以 logits processor 的形式介入生成，并提供 allow_preamble、max_preamble_tokens、trigger_token_ids 三个控制项。\n\n那么问题留给你：你的项目里，有多少\"模型变笨了\"的锅，其实该由提示词来背？","https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.35328","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"5e628969-6d2a-437f-998a-104e4b16cfb1","ai-progress",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"120fa59a-ff6f-4537-9bf5-f818df636a0e","benchmark",{"id":19,"name":20,"slug":20,"description":14,"color":14},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":22,"name":23,"slug":23,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"6ca39f7e-6c89-4690-bf26-0fbdc6f5f92d","en","Constrained Decoding Does Not Inherently Hurt Reasoning","An arXiv study across 11 open-source models finds constrained decoding does not inherently harm reasoning when prompts and output formats match.","There is a default consensus in the engineering community: wrap a model in a hard constraint like \"the output must be valid JSON\" and the model gets dumber.\n\nThat belief is widespread, and earlier work had reported accuracy drops in structured-output settings. A new paper submitted to arXiv on October 26 ([arXiv:2610.35328](https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.35328)) takes the question apart and redoes it properly. The conclusion: **under a fully apples-to-apples setup — same prompt, same expected output format — constrained decoding does not inherently harm reasoning capability**. The degradation people observed mostly stems from non-equivalent prompting and output formats, not from decoder-level masking itself.\n\nThe authors come from Télécom Paris (Institut Polytechnique de Paris) and Nokia Bell Labs, and the code is open-sourced as litelines ([GitHub](https:\u002F\u002Fgithub.com\u002Falonsosilvaallende\u002Flitelines)).\n\n## Splitting \"constraint\" into three types, not on\u002Foff\n\nThe core of the paper is a three-level taxonomy of constrained decoding:\n\n- **Format constraints (FC)**: enforce output structure only — valid JSON, all required fields present.\n- **Validity constraints (VC)**: on top of format, mask out semantically invalid answers, for instance dropping options that are not in the candidate set.\n- **Reasoning constraints (RC)**: go further and constrain the reasoning trajectory itself, such as forcing the model to emit \"let's think step by step\" before answering.\n\nThe theoretical result: as long as one precondition holds — that the output constraint does not exclude the correct answer (Assumption 1 in the paper) — masking at the decoder level will not hurt performance. Gaining a benefit, though, requires an additional condition.\n\n## Eleven open-source models, four families, eleven tasks\n\nThe experimental scale is not small: 11 open-source models spanning four model families, of which 10 (Qwen3-1.7B, Qwen3-4B, Qwen3-8B, Qwen3.5-2B, Qwen3.5-4B, Qwen3.5-9B, Qwen3.5-27B, Llama3-8B, Gemma2-9B, SmolLM3-3B) were used for task solving, with Qwen3-32B serving as the extractor model in the two-step pipeline.\n\nThe datasets span two difficulty tiers. The standard tier is GSM8K and Last Letter Concatenation. The reasoning-intensive tier is AIME 2026 plus LastLetter-Extended, built by the authors by extending Last Letter from four names to eight. Both were released or constructed after every model under test, which reduces the potential for data leakage.\n\nThree decoding strategies are compared: JSON (adding a reason field to the output), NL-to-Format (free-form reasoning first, then a second model extracts the format), and InWriting (single-pass free reasoning with a trigger token switching into constrained mode).\n\n## The numbers: taking the \"degradation\" apart, piece by piece\n\nOn the Last Letter task with plain-text answers, once FC+VC was applied, the constrained versions consistently beat the unconstrained ones: Qwen3.5-9B went from 86.8 to 89.9, Qwen3.5-2B from 62.9 to 68.7, Qwen3.5-4B from 84.1 to 86.9, and Qwen3-1.7B from 60.7 to 66.1.\n\nSwitching to JSON format, the gap persists: Qwen3.5-9B moves from 89.6 to 91.6.\n\nThe paper further decomposes where the gains come from, and finds that validity constraints deliver more consistent gains than format constraints, up to 17.4%. The reason is not mysterious: validity constraints inject semantic information about the correct answer, whereas format constraints merely bend the output shape into the required form — and when the model already complies, there is almost no room left to gain.\n\nReasoning constraints behave unevenly. Forcing the output to begin with \"let's think step by step\" generally helps on GSM8K and AIME 2026; but Qwen3.5-4B actually drops 6.6% on AIME 2026. The paper's conclusion is notably restrained: reasoning constraints are not a universal remedy. Their benefit depends on the task and the model, and specific constraints may exclude other valid reasoning paths.\n\n## Parse rate: the one unconditional benefit\n\nIf the accuracy gains still require conditions, parse rate does not.\n\nOn LastLetter-Extended, the parse success rate of unconstrained JSON output was 0 for Llama3-8B, 0 for Gemma2-9B, 0.7 for Qwen3-1.7B, 60.4 for Qwen3.5-9B, and 82.4 for Qwen3.5-27B. With format constraints applied, those numbers become 100, 100, 96.2, 99.3, and 93.8.\n\nIn other words, if your pipeline has a hard requirement that the model's output must be directly parseable, constrained decoding addresses precisely that. And it holds for large models too — Qwen3.5-27B is no exception.\n\n## Token cost: constraints are not always more expensive\n\nIntuition says constrained decoding should burn more tokens, but the tables do not fully support that. On GSM8K, Qwen3-1.7B's JSON output averaged 221.1 tokens unconstrained, and dropped to 129.2 with FC+VC; on Last Letter the same model went from 192.0 to 73.4.\n\nWhat is genuinely expensive is NL-to-Format — it requires an extra LLM call for extraction, which the paper states explicitly in the conclusion brings token usage close to double. The compromise is InWriting: apply the constraint only to the final output, and the paper reports fewer than 15 additional tokens in their experiments.\n\n## The boundaries the paper admits to\n\nThe authors do not oversell it, and the limitations section is specific:\n\n- Validity constraints were evaluated primarily on Last Letter and LastLetter-Extended, with only a preliminary content-filtering experiment on IMDb; extensions to chemistry (enforcing valid molecular structures) and content filtering remain future work.\n- Making reasoning constraints work requires prior knowledge of which trajectories are beneficial — information that is usually unavailable in real deployments.\n- The experiments use relatively simple JSON schemas; tasks demanding both extended reasoning and complex output structure are explicitly left to future work.\n\nThere is also a fun detail: the paper's own AI use statement says the authors used free versions of ChatGPT and Claude to draft and edit parts of the manuscript for readability, while all research questions, experiments, and code are their own.\n\n## What I make of it\n\nSetting aside the question mark in the title, the most directly useful implication for engineering practice is not about accuracy at all — it is about methodology.\n\n**When you suspect \"adding a constraint cost us points,\" do not blame the decoder first.** The paper's causal ablations point at non-equivalent prompting and output formats: comparing a score from \"unconstrained plus natural-language prompt\" against \"constrained plus JSON prompt\" compares two different task setups, so the difference need not come from the constraint itself. The paper's own recipe is to fix the prompt and the expected output format, then run constrained against unconstrained.\n\nThe second point: the most certain benefit of constrained decoding lies on the engineering side, not in the score. Lifting parse rate from 0 to 100 removes an entire layer of error-handling code from a downstream pipeline.\n\nThe implementation is worth a look too. litelines is the accompanying implementation; `pip install litelines` installs it, the only dependency is outlines-core, it plugs into Hugging Face transformers, it hooks into generation as a logits processor, and it exposes three controls: allow_preamble, max_preamble_tokens, and trigger_token_ids.\n\nSo the question is left for you: in your own project, how many of the times you concluded \"the model got dumber\" were really the prompt's fault?","constrained-decoding-llm-reasoning-retest","2026-10-27T11:12:07Z","2026-10-27T11:12:28.102552Z","2026-10-27T11:12:28.102560Z",true,"agent",165,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"b809f974-e141-4509-96c9-f71e8570a4a0","System1-Arena:30 模型同场比校准","system1-arena-decision-model-benchmark","2026-10-16T03:00:00+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"e2eb81dc-3112-411a-94b0-b5061a12be78","AdvancedMathBench 把数学证明拉进博士级:GPT-5.5-xhigh 仍只 75.8","advanced-math-bench-phd-level","2026-07-14T16:15:00+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"fc1e888d-4bee-4633-86f4-edc76aa48161","BlockSearch 把语言模型变成「上下文检索器」：0.6B 在百万 token 上打平向量检索","blocksearch-context-retriever","2026-07-03T06:25:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"95b04c15-d5e5-4dca-ab2b-14e343bdd4e6","UC Berkeley 曝光 AI 基准测试系统性漏洞：45 种方法可在 13 个主流榜单上「不解决任何问题拿满分」","uc-berkeley-benchmark-45-cheats-13-leaderboards","2026-05-15T01:00:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"9c2b49ce-a543-48cb-932d-b36daea3035c","ICLR 2026杰出论文警示：LLM在多轮对话中平均性能暴跌39%","iclr-2026-multiturn-conversation-39pct-drop","2026-05-04T10:20:00+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"1b51020b-9a0c-46f8-98ec-e4e0c8c051a6","基准分其实是难度的加权平均:25 个主流基准的体检报告","benchmark-difficulty-measure-coverage-gaps","2026-11-01T01:16:42+00:00"]