[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-output-format-silent-errors-small-parser":3,"topics-all":38,"news-related-0be3d0c4-dee7-4fc2-9364-6a8ce68f5f3d":67},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"0be3d0c4-dee7-4fc2-9364-6a8ce68f5f3d","0.8B 模型换输出格式,静默错误率 10.3%→4.5%","arXiv 论文把同一棵订单树写成六种无损编码,在两个亚十亿模型上跑 300 次微调,结论是决定静默错误率的不是输出长度而是表面形式:改用普通词后准确率打平,静默错误率从 10.3% 降到 4.5%,token 少 4.1 倍,代价是严格拒答会误伤 2.1% 到 10.1% 的真实订单。","结构化输出里有一种错误特别阴险：不是模型吐出乱码，而是它吐出一棵语法完全合法的树——里面写着一个顾客根本没说的配料。下游检查格式、检查字段、跑通业务逻辑，订单照样错下去。这类错误对调用方完全不可见，论文把它叫静默错误率。\n\n11 月 12 日提交到 arXiv 的一篇论文（[arXiv:2611.17169](https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.17169)，ITMO 大学 IDM 实验室）把这件事做了量化。它不研究怎么把模型训得更强，而是研究一个完全免费的设计变量：你让模型输出什么格式。\n\n## 六种格式，同一棵树\n\n实验用 PIZZA 披萨点单语义解析基准：训练池 2,456,446 条由语法生成，开发集 348 条，测试集 1,357 条由人手写。论文挑了两个参数量不到十亿的模型——Qwen3.5-0.8B 和以卷积为主的 LFM2.5-350M——用 LoRA 微调，跑满 300 次（6 种格式 × 5 个数据池规模 × 5 个随机种子 × 2 个模型）。\n\n设计很干净：六种编码全部无损，可解码回同一棵树，长度从 44.3 token 排到 10.9 token。更关键的是作者留了两对长度几乎一样的对照——exr 是数据集原生的 s-表达式，json 是长度匹配的括号记法；dsl 是符号速记（+cheese），words 是同结构的普通词写法（extra cheese）。后两者 token 数只差不到一个，唯一区别是一个写符号、一个写人话。\n\n## 长度不是机制，表面形式才是\n\n直觉是输出 token 越少，留给出错的余地越小。但论文把长度对准确率做回归，10 个模型×规模组合里 10 个的斜率都是正的（去掉 token 反而掉分），每 10 token 最多掉 1.1 分，而且只有 1 个组合统计上显著。更狠的反例是 exr-short——它只是把 exr 的构造器名换成单字母，长度从 44.3 砍到 32.3 token，Qwen 上准确率直接掉 10.6 分且在每个数据规模上都显著；而长度只有 exr 一半的 tagged 格式，在 Qwen 上却和 exr 打平。长度本身不预测准确率。\n\n真正的胜负在长度匹配的那一对：words 在全部 10 个模型×规模格子里都赢过 dsl，Holm 校正后 9 个显著，全量数据下 Qwen 上领先 7.8 分（区间 6.1 到 9.5）、LFM2 上领先 5.4 分。反直觉的是在 LFM2 上，words 反而比 dsl 多花 token（12.0 对 11.4），仍然更准——更长却更准，说明它赢的不是短，是词。\n\n至于跟原生格式比，论文自己的读法很克制：Qwen 上 87.5% 对 85.8%，准确率是打平而不是超越，Holm 校正后在 Qwen 上并不显著，作者把它写成「用 4.1 倍更少的 token 换来同样的准确率」。\n\n## 错在哪里：否定\n\n论文还换了个评估口径。传统做法看精确匹配率，但部署时真正该盯的是静默错误率——能解析、但内容错的输出比例。四分账是：正确 + 静默错 + 解析失败 + 拒答 = 1。只有后两种对调用方可见，可以重试或转人工；静默错直接到达顾客。\n\nwords 格式在 Qwen 上的静默错误率是 4.5%，原生 s-表达式是 10.3%，比值 2.3；LFM2 上是 10.0% 对 17.9%，比值 1.8。这组数字是论文实测口径，模型换成更大的、任务换成别的都还待验证。更有意思的是错的内容变了：Qwen 上 words 的静默错误里否定只占 5%，其他格式是 19% 到 34%；换算成订单，Qwen 用 exr 时 2.0% 的订单会出现静默的否定错误，用 words 只有 0.2%。因为 words 直接写「不要洋葱」，而其他格式要把它翻译成 NOT 结构或符号。另一个细节：words 凭空编造顾客没提的配料少 5.5 倍。\n\n## 代价也要算清楚\n\n论文没停在好数字上。守门策略（拒答非订单输入、听懂口头改单）在留出集上的拒绝率，Qwen 是 95.9% 到 99.0%，纠错精确匹配是 62.9% 到 75.3%，那个经典的「不要芝士」案例在 60 次全量跑里全部修好。\n\n但严格拒答有明码标价的损失：真实订单里有 2.1% 到 5.4%（Qwen）、6.3% 到 10.1%（LFM2）被误拒。被拒的几乎全是训练语法里从不出现的礼貌用语——带问候、带 please、带 thanks 的 289 条测试订单，words 格式的误拒率是 6.1%，不带的是 2.6%。格式能救幻觉，但救不了没见过的礼貌。\n\n延迟是纯赚的：Qwen 上 words 比 exr 快 3.7 倍，LFM2 上 4.4 倍，显存基本不动（1,704 到 1,806 MiB）。附录还有一段实机演示：Q8_0 量化、795 MiB 文件、350 MiB 常驻，iPad 上走 llama.cpp 加 Metal，单条订单中位 167 毫秒——作者注明这只是 10 单的演示，不是评测。\n\n## 边界在哪，能不能外推\n\n诚实的边界：只有一个数据集；两个模型都不到十亿参数，论文自己说这个效应可能随规模缩小；守门数据是作者自己生成的；导出的 LFM2 GGUF 和训练时只对上 93% 的订单。\n\n但机制层有独立佐证。EMNLP 2024 的《Let Me Speak Freely?》（[ACL Anthology](https:\u002F\u002Faclanthology.org\u002F2024.emnlp-industry.91\u002F)）在另一支团队、另一批模型上发现，把输出限制成结构化格式会明显拖垮推理能力，而且限制越严格退化越大；本篇论文的综述里也把「强迫模型输出 JSON 会牺牲推理准确率」列为已有结论。LREC 2026 的《Format Matters》（[HAL](https:\u002F\u002Fhal.science\u002Fhal-05546569v1)）在 13 个开源模型、4 个槽位填充和 3 个命名实体任务上，测出 2 到 46 个 F1 点的格式摆动。三支不同团队、三个不同来源，指向同一件事：输出格式不是显示层细节，它会改写模型的行为。\n\n## 所以呢\n\n输出格式是少数不用重训、不用加数据、不花推理成本就能拿到的杠杆。但先量你自己的静默错误率，再动格式——多数团队手上只有准确率，而准确率恰恰是那个会骗人的指标。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.17169","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"5e628969-6d2a-437f-998a-104e4b16cfb1","ai-progress",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"120fa59a-ff6f-4537-9bf5-f818df636a0e","benchmark",{"id":19,"name":20,"slug":20,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",{"id":22,"name":23,"slug":23,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"8922ecae-9467-4441-af7c-68ff7eae217c","en","Plain Words Roughly Halve Silent Errors in a 0.8B Parser","Plain words cut silent errors from 10.3% to 4.5% in 300 runs, using 4.1x fewer tokens, but strict refusal rejects up to 10.1% of real orders.","One kind of failure in structured output is especially nasty: the model does not emit garbage, it emits a syntactically perfect tree that contains a topping the customer never ordered. The pipeline checks the format, checks the fields, runs the business logic, and the wrong order still goes through. Nothing downstream can see it. The paper calls this the silent error rate.\n\nA paper submitted to arXiv on 12 November ([arXiv:2611.17169](https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.17169), from the IDM Lab at ITMO University) puts numbers on it. It is not about how to train a stronger model. It is about a completely free design variable: what format you ask the model to write in.\n\n## Six formats, one tree\n\nThe experiment uses the PIZZA pizza-ordering semantic parsing benchmark: a training pool of 2,456,446 utterances generated from a grammar, a 348-utterance development set, and a 1,357-utterance test set written by people. The authors picked two sub-billion-parameter models, Qwen3.5-0.8B and the largely convolutional LFM2.5-350M, fine-tuned them with LoRA, and ran 300 configurations in total (6 formats x 5 training pool sizes x 5 seeds x 2 models).\n\nThe design is clean: six encodings, all lossless, each decoding back to the identical tree, ranging from 44.3 tokens per order down to 10.9. What matters more are the two matched-length control pairs the authors kept. `exr` is the dataset's native s-expression and `json` is a bracket notation of about the same length. `dsl` is a terse symbolic grammar using sigils such as `+cheese`, and `words` has the same structure with the markers written as ordinary words, `extra cheese`. The latter two differ by less than a token; the only difference is whether a marker is a symbol or a word.\n\n## Length is not the mechanism, surface form is\n\nThe intuition is that fewer output tokens leave less room for error. But when the paper regresses accuracy on output tokens, the slope is positive (removing tokens lowers accuracy) in 10 of 10 model-size cells, never more than 1.1 points per ten tokens, and it separates in only one cell. The sharper counterexample is `exr-short`: it only renames `exr`'s constructors to single letters, cutting length from 44.3 down to 32.3 tokens, and Qwen loses 10.6 points of exact match, separating at every pool size. Meanwhile `tagged`, at half the length of `exr`, matches it on Qwen. Length does not predict accuracy.\n\nThe real contest is in the matched-length pair: `words` beats `dsl` in all 10 model-size cells, separating in 9 after Holm correction, leading by 7.8 points on Qwen at full pool (interval 6.1 to 9.5) and 5.4 on LFM2. What is counterintuitive is that on LFM2 `words` actually emits more tokens than `dsl` (12.0 against 11.4) and is still more accurate. Longer and more accurate means it is not winning by being short, it is winning by being words.\n\nAgainst the native format the authors stay deliberately restrained: 87.5% against 85.8% exact match on Qwen is a match, not a win, and after Holm correction it does not separate on Qwen. They write it as matching the native format's accuracy with 4.1x fewer tokens.\n\n## Where the errors come from: negation\n\nThe paper also changes the evaluation lens. The usual metric is exact match, but what a deployment should actually watch is the silent error rate, the share of outputs that parse cleanly and are still wrong. The budget splits four ways: correct + silent + unparseable + refused = 1. Only the last two are visible to the caller, so they can be retried or handed to a human. Silent errors reach the customer.\n\nOn Qwen the `words` format has a 4.5% silent error rate against 10.3% for the native s-expression, a ratio of 2.3; on LFM2 it is 10.0% against 17.9%, a ratio of 1.8. These are the paper's own measurements on sub-billion models, and whether they hold on larger models or other tasks is still open. More interesting is what changed in the errors. On Qwen, negation accounts for only 5% of `words`' silent errors against 19% to 34% for the other formats. Expressed as a rate per order, Qwen makes a silent negation error on 2.0% of orders with `exr` and 0.2% with `words`. The reason is visible in the output: `words` writes `no onions` the way customers say it, while the other formats translate it into a NOT construct or a sigil. One more detail: `words` invents entities the customer never mentioned 5.5x fewer times on Qwen.\n\n## The bill comes due\n\nThe paper does not stop at the good numbers. The guardrail trained into every run (refusing off-menu input, applying in-utterance self-corrections) refuses held-out negatives 95.9% to 99.0% of the time on Qwen, and exact match on corrections runs 62.9% to 75.3% on Qwen. The classic cheese failure, where a customer says `a pizza with cheese, wait no cheese, only tomato sauce`, is fixed in 60 of 60 full-pool runs.\n\nBut strict refusal has a price, and it is priced explicitly: 2.1% to 5.4% of real test orders are refused on Qwen, and 6.3% to 10.1% on LFM2. Reading them, they are genuine orders in human phrasing that the synthetic grammar never produces. Almost every refused order contains a word the training grammar never uses. An order with a greeting, with `please` or with `thanks` (289 of the test orders) is refused 6.1% of the time by Qwen `words` against 2.6% for other orders. Format rescues you from hallucination; it does not rescue you from politeness it never saw.\n\nLatency is pure profit: `words` answers 3.7x faster than `exr` on Qwen and 4.4x on LFM2, and GPU memory barely moves, between 1,704 and 1,806 MiB. The appendix adds a device demonstration: Q8_0 weights, a 795 MiB file, 350 MiB resident, running on an iPad through llama.cpp with Metal offload, at a median of 167 ms per order. The authors note this is a demonstration on ten orders, not an evaluation.\n\n## Boundaries, and whether it travels\n\nThe honest limits: one dataset; both models are under a billion parameters and the paper itself says the effect may shrink with scale; the guardrail data was generated by the authors; and the exported LFM2 GGUF agrees with its trained run on only 93% of orders.\n\nBut the mechanism has independent corroboration. The EMNLP 2024 paper 'Let Me Speak Freely?' (on [ACL Anthology](https:\u002F\u002Faclanthology.org\u002F2024.emnlp-industry.91\u002F)) reports, from a different team and a different set of models, that restricting output to structured formats degrades reasoning, and that stricter constraints degrade it more. This paper's related-work section lists 'forcing an LLM into JSON costs reasoning accuracy' as an existing result. The LREC 2026 paper 'Format Matters' (on [HAL](https:\u002F\u002Fhal.science\u002Fhal-05546569v1)) measures output format as a first-class hyperparameter across 13 open-weight models, 4 slot-filling and 3 NER benchmarks, and finds statistically significant swings of 2 to 46 F1 points. Three different teams, three different sources, the same conclusion: output format is not a display detail, it changes how the model behaves.\n\n## So what\n\nOutput format is one of the few levers you can pull without retraining, without more data, and without paying anything at inference time. But measure your own silent error rate before touching the format. If all you track is exact match, then exact match is the metric that will mislead you.","output-format-silent-errors-small-parser","2026-11-13T09:20:00Z","2026-11-13T09:14:35.863875Z","2026-11-13T09:14:35.863883Z",true,"agent",194,[39,49,58],{"slug":40,"tag_slug":41,"title_zh":42,"title_en":43,"intro_zh":44,"intro_en":45,"id":46,"is_active":35,"created_at":47,"modified_at":48},"image-generation","text-to-image","图像生成：从文生图到逐框编辑的模型竞赛","Image Generation: from text-to-image to frame-level editing","文生图已是红海，战场正转向编辑与可控性：FLUX.2\u002F3 用开源和量化把生图搬进消费级显卡，Qwen-Image、Seedream 等国产模型紧咬不放，GPT Image 2 则把推理能力带进像素。本专题追踪图像生成与编辑模型的发布、架构演进、加速量化与商业化数据，看「好图」如何变成「可用的图」。","Text-to-image is a crowded race, and the frontier has moved to editing and control: FLUX.2\u002F3 push open weights and quantization onto consumer GPUs, Qwen-Image and Seedream close in from China, and GPT Image 2 brings reasoning into pixels. This topic tracks model releases, architecture shifts, acceleration work, and the business numbers behind \"pretty\" becoming \"usable\".","52de7c59-7864-4569-8ae7-3d656d26a1a1","2026-11-20T07:20:23.534506Z","2026-11-20T07:20:23.534527Z",{"slug":50,"tag_slug":50,"title_zh":51,"title_en":52,"intro_zh":53,"intro_en":54,"id":55,"is_active":35,"created_at":56,"modified_at":57},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":59,"tag_slug":59,"title_zh":60,"title_en":61,"intro_zh":62,"intro_en":63,"id":64,"is_active":35,"created_at":65,"modified_at":66},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":68},[69,74,79,84,89,94],{"id":70,"title":71,"news_slug":72,"published_at":73},"8e741f04-82a2-46da-9d30-3c31117ca70c","Ambient SFT:让预训练的自己当高温老师","ambient-sft-temperature-matching","2026-11-18T17:30:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"08c85d0c-1ad8-43d6-bd32-0ec8b21bf2aa","给教师开小抄,学生涨分却不是抄来的","opsd-privileged-solution-teacher-audit","2026-11-15T01:30:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"a6d8e95f-41ba-4a97-815c-94eff9401706","按步数赢按算力输:GRPO密集奖励的评测账本","grpo-dense-rewards-matched-compute","2026-11-11T05:05:00+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"e8168e92-d16e-452a-991c-aac74dd23ec0","预训练赢了,微调输了:Muon 的反转","muon-adamw-fine-tuning-gap-reversal","2026-11-04T01:12:00+00:00",{"id":90,"title":91,"news_slug":92,"published_at":93},"bb244989-00b5-4b67-b70a-5316452ee44e","把 RLVR 的\"零梯度\"浪费变成有效信号：Mistral 联合剑桥用 Reasoning Arena 让推理训练提速 41%","reasoning-arena-mistral-cambridge-41pct","2026-06-08T00:00:00+00:00",{"id":95,"title":96,"news_slug":97,"published_at":98},"42f5ea83-365c-4a4e-a318-17d7c4c11c15","论文实测:自洽性证书在AIME上24%的题上会误触发","self-consistency-certificates-misfire-correlated-reasoning","2026-11-21T09:30:00+00:00"]