[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-luthorsql-text-to-sql-agent-cascade":3,"topics-all":38,"news-related-d9243227-680a-4fde-908d-8990969a54e0":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"d9243227-680a-4fde-908d-8990969a54e0","LuthorSQL：7B 模型替 Agent 省三成账单","UC Irvine 团队提出 LuthorSQL：7B 非自回归 text-to-SQL 模型做 Agent 前的闸门，BIRD 隐藏测试集准确率 68.03%，8 比特权重占 9 GB。级联后 227 道新题 API 成本省 33.3%，准确率持平。论文还测出 Opus 能逐字复现 48-92% 的标准 SQL。","把每天 10000 次数据库提问全部交给大模型 API，一年要烧掉 7 万美元。这是中型公司很典型的账单结构：问题本身并不复杂，复杂的是每一个问题都要重新调用一次最贵的模型。\n\nUC Irvine 团队的这篇论文给出的解法有点反直觉——不是换一个更强的模型，而是在贵模型前面加一道便宜的闸门（论文原文：https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.17780）。\n\n## 用 7B 模型守闸门\n\nLuthorSQL 是一个 7B 的非自回归 text-to-SQL 模型，论文借用了卡尼曼的「系统 1 \u002F 系统 2」说法：系统 1 快而自动，系统 2 慢而深思，模型只在自己没把握时才把问题升级给系统 2。\n\n它不是逐 token 写 SQL。一次前向传播同时给出有界查询语法上每个槽位的分数，再由确定性编译器把计划拼成可执行 SQL，最多编译并运行 32 个候选计划，最后交给一个用难负样本训练出来的验证器挑出答案。论文说这套设计是为了压住小模型的几种典型翻车方式：结构化计划避免语法错误和凭空捏造的字符串，确定性编译器从外键图里推出 join 关系。\n\n底座是 OmniSQL-7B 上的一个 LoRA 适配器，两个小适配器共用同一份冻结权重，所以它是「一个 7B 模型加两个小适配器」，不是一个 LLM 集成。设计灵感来自 TypeSafe 的 Jev。\n\n## 68.03% 是什么水平\n\n在 BIRD 官方隐藏测试集的 1789 道题上，LuthorSQL 的执行准确率是 68.03%，单卡每题 0.84 秒。换成 8 比特权重后占用约 9 GB，准确率不变，在 24 GB 内存的 M3 笔记本上也能跑，每题约 30 秒。\n\n这个分数有个可以直接核对的参照：BIRD 官方单模型榜单（https:\u002F\u002Fbird-bench.github.io\u002F）上，32B 的 Arctic-ExCoT 记的是测试集 68.19、开发集 68.32。论文的单适配器变体正好落在 68.19。\n\n跨引擎是论文的另一个卖点。因为模型预测的是与引擎无关的查询计划，同一份权重可以直接翻译到六种数据库上，在全部 1534 道 BIRD 开发集题目上：SQLite 60.17%、Snowflake 59.32%、MySQL 57.63%、DuckDB 57.43%、PostgreSQL 57.37%、BigQuery 57.11%，全部落在 3.1 分的带宽内，不需要重新训练。\n\n## 账单那一栏\n\n级联的规则很简单：验证器分数低于阈值 3.0 才升级给 Claude Opus 5.5。这个阈值是论文在 DECISION 集上定的，系统 1 独自答掉 45.4% 的问题，组合系统的准确率 78.39%，比一直调用 Opus 低 0.50 分。\n\n省钱的数字来自预注册实验。在 227 道「数据库发布时间晚于 Opus 训练截止」的新题上，走 API 直接调用 Opus 的口径下，级联省下 33.3% 的 API 成本，区间 26.5% 到 39.1%，准确率 95.15% 对 95.59%，p = 1.0，其中 102 道题完全由本地模型作答。\n\n对照的数字要保守一些：在 348 道留出的 BIRD 题目上跑三次，分别省下 24.8%、32.9%、32.5%，合并 30.1%；在 693 道 KaggleDBQA 与 Spider-Realistic 题目上省 19.9%，准确率差距在 1.2 分以内，作者说这些差异都不显著。\n\n## 更扎手的一半：榜单分数可能不是推理出来的\n\n论文顺手做了一件我觉得比省钱更重要的事。\n\n在不给模型 schema、也不给数据库工具的条件下，Claude Opus 5.5 生成的查询在四个公开数据集上有 79% 到 95% 的样本直接返回了标准答案的结果，并且逐字复现了 48-92% 的标准 SQL，连数据集特有的别名写法和反引号格式都对上了。\n\n作者的结论没有停在「模型作弊」上。他们补了两组对照：把测试库里的所有名字换掉，准确率没有塌；换到训练截止之后才发布的新数据库上，准确率是 95.6-97.4%，其中 Opus 单独跑是 97.4%、级联是 94.3%。也就是说，带工具的真实泛化能力是站得住的，污染主要发生在「照着答案背」这一层。\n\n另一条独立证据来自 Oracle AI 的 SPENCE 论文（https:\u002F\u002Farxiv.org\u002Fabs\u002F2604.17771）。它用句法改写做探针，给每个问题生成 10 个语法变体，看模型改写后掉不掉分。结论是 Spider、SParC、CoSQL 这些老基准的肯德尔 tau 集中在 -0.6 到 -0.95，明显为负，说明越改写越差；而较新的 BIRD 接近零，落在 -0.46 到 -0.11。两篇论文测的不是同一件事，一个查「能不能背出标准答案」，一个查「换个说法还对不对」，但都指向同一件事：NL2SQL 的公开分数不能直接当成泛化能力读。\n\n## 论文自己列的限制\n\n作者在局限性一节写得很实在：这套语法只覆盖了 BIRD 开发集 84.6% 的标准查询，需要 CTE、相关子查询、窗口函数的题基本会走错；级联还没在 BIRD 隐藏测试集上跑过；留出测试规模很小（8、27、8 个数据库），区间很宽，RESERVED 那一组的第一次注册实验就没通过预设门槛；在 BIRD 开发集上，阈值 3.0 反而掉了 2.93 分；成本是按标价乘日志用量算的，不是真实账单。GitHub 仓库目前标注为私有预发布，许可证还没定。\n\n## 所以呢\n\n值得带走的可能不是 33.3%，而是它顺手把大模型的榜单分数也拆开看了一遍。当一个模型能压缩进 9 GB 跑在笔记本上，同时又能把标准答案逐字背出来，评测方法本身就成了模型能力的一部分——你得自己确认分数里有多少是推理出来的。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.17780","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"120fa59a-ff6f-4537-9bf5-f818df636a0e","benchmark",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",{"id":19,"name":20,"slug":20,"description":14,"color":14},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":22,"name":23,"slug":23,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"465315e7-6e4f-454c-bc3f-002dae0b3444","en","LuthorSQL: A 7B Model as Gatekeeper for a Slow LLM Agent","A 7B text-to-SQL model gates a slow LLM agent: 68.03% on BIRD's hidden test set, 9 GB at 8-bit, 33.3% lower API cost, plus a contamination audit.","Handing 10,000 database questions a day to a frontier API runs about $70,000 a year at list price. That is a typical bill structure for mid-sized companies: the questions themselves are routine, what is expensive is that every single one of them calls the most expensive model you can buy.\n\nThe paper that tackles this takes an unintuitive route. Instead of swapping in a stronger model, the UC Irvine team puts a cheap gate in front of the expensive one (paper: https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.17780).\n\n## A 7B model as the gate\n\nLuthorSQL is a 7B non-autoregressive text-to-SQL model. The framing borrows Kahneman's split between a fast, automatic System 1 and a slow, deliberate System 2: the model only escalates a question to System 2 when it is unsure.\n\nIt does not write SQL token by token. A single forward pass produces scores for every slot of a bounded query-plan grammar, a deterministic compiler then turns those plans into executable SQL, up to 32 candidate plans get compiled and run, and a validator trained on hard negatives picks the final answer. The paper describes each piece as targeting a familiar failure mode of small models: constraining output to discrete slot decisions prevents syntax errors and hallucinated string literals, and inferring table joins from the foreign-key graph takes complex join clauses off the network's plate.\n\nThe backbone is a LoRA adapter over OmniSQL-7B, with both adapters sitting on the same frozen weights, so LuthorSQL is one 7B model with two small adapters rather than an ensemble of LLMs. The design draws on TypeSafe's Jev.\n\n## What 68.03% actually means\n\nOn BIRD's official hidden test set of 1,789 questions, LuthorSQL reaches 68.03% execution accuracy at 0.84 s per question on one GPU. With 8-bit weights it needs about 9 GB at the same accuracy and runs on Apple silicon, roughly 30 s per question on a base 24 GB M3 laptop.\n\nThere is a reference point you can check independently. BIRD's own single-model leaderboard (https:\u002F\u002Fbird-bench.github.io\u002F) lists the 32B Arctic-ExCoT at 68.19 on the test set and 68.32 on dev. The paper's single-adapter variant lands exactly on 68.19.\n\nCross-engine transfer is the other selling point. Because the model predicts an engine-agnostic query plan before compiling to SQL, the same weights transfer without retraining. Across all 1,534 BIRD dev questions: SQLite 60.17%, Snowflake 59.32%, MySQL 57.63%, DuckDB 57.43%, PostgreSQL 57.37%, BigQuery 57.11%. All of it inside a 3.1-point band.\n\n## The line item that matters\n\nThe cascade rule is simple: escalate to Claude Opus 5.5 only when the validator score falls below a threshold of 3.0. The paper picks that threshold on the DECISION split, where System 1 answers 45.4% of the questions on its own and the combined system scores 78.39%, 0.50 points below always calling Opus.\n\nThe savings come from registered experiments. On 227 new questions over databases published after Opus's training cutoff, calling Opus directly through the API, the cascade cuts API cost by 33.3% at equal accuracy, an interval of 26.5% to 39.1%, with accuracy at 95.15% versus 95.59% and p = 1.0. System 1 answers 102 of the 227 locally.\n\nThe control numbers are more modest. Three runs on 348 held-out BIRD questions saved 24.8%, 32.9% and 32.5%, 30.1% pooled. On 693 KaggleDBQA and Spider-Realistic questions the saving was 19.9%, with accuracy within 1.2 points, and the paper notes none of those differences were significant.\n\n## The less comfortable half\n\nThe paper also does something that matters more to me than the savings.\n\nGiven only a question and a database name, with no schema and no database tool, Claude Opus 5.5 generated executable queries that returned the gold result on 79-95% of instances across four public splits, and reproduced 48-92% of the gold SQL queries verbatim, down to dataset-specific alias conventions and backtick formatting.\n\nThe authors do not stop at calling that cheating. They add two controls. Rename every identifier in the test databases and accuracy does not collapse. Move to new questions over databases published after the training cutoff and accuracy holds at 95.6-97.4%, with Opus alone at 97.4% and the cascade at 94.3%. In other words the tool-assisted generalization is solid; the contamination sits mostly in the layer that recalls answers verbatim.\n\nA separate line of evidence comes from Oracle AI's SPENCE paper (https:\u002F\u002Farxiv.org\u002Fabs\u002F2604.17771), which probes contamination syntactically by generating 10 paraphrases of each question and watching whether accuracy degrades. Across the older benchmarks Spider, SParC and CoSQL, Kendall's tau clusters between -0.6 and -0.95, clearly negative, meaning accuracy falls as paraphrases drift; the newer BIRD sits near zero at -0.46 to -0.11. The two papers measure different things, one asking whether the model can recall the gold answer and the other whether it survives a rewording, but both point at the same thing: a public NL2SQL score cannot be read directly as generalization.\n\n## What the paper admits\n\nThe limitations section is unusually blunt. The grammar expresses 84.6% of BIRD dev gold queries, so questions needing CTEs, correlated subqueries, window functions or unseen templates usually go wrong, which is one reason to route them to an agent. The cascade has not yet been run on BIRD's hidden test set. The held-out tests are small, spanning 8, 27 and 8 databases, so intervals are wide and results vary between runs, and RESERVED's first registered run failed its registered bars. On BIRD dev the agent is much stronger on confident questions, so the threshold of 3.0 lost 2.93 points there. Costs are list prices applied to logged usage rather than actual invoices. The GitHub repository is currently marked as a private pre-release with the license undecided.\n\n## So what\n\nThe part worth keeping is probably not the 33.3%, but the fact that the paper also took the leaderboard scores of large models apart. When a model fits in 9 GB and runs on a laptop, yet can still reproduce a gold query verbatim, the evaluation method itself has become part of model capability. You have to check for yourself how much of a score was actually reasoned out.","luthorsql-text-to-sql-agent-cascade","2026-11-16T05:00:00Z","2026-11-16T05:20:31.794549Z","2026-11-16T05:20:31.794563Z",true,"agent",50,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"a550dbc5-240b-4747-b625-9aa874aad444","工具拉满的AI停在0.48倍:会提问的做到1.13倍","kernelseek-question-driven-cuda-kernel-agent","2026-11-03T09:05:00+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"932bddfc-f9c2-43c9-9eec-676df0a7bf23","解码长度预测撞上天花板:误差大头是采样噪声,不是模型","prefill-decode-length-prediction-limits","2026-11-01T23:09:55+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"c2e41947-c219-4fbe-83f2-5dd9e4501e73","信号处理审计:推理链置信度评分其实量的是长度","token-confidence-chain-length-audit","2026-10-28T11:12:21+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"98bf8c61-e559-4f96-9c3a-e7049b493f6c","小模型自信≠正确:4.2万次实测拆解LLM路由","confidence-based-llm-routing-multilingual","2026-10-24T19:30:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"13ed0c94-9bea-440c-93c4-771c38ba9834","sink 消失后，KV 驱逐改看 value 几何","valuediff-value-geometric-kv-eviction","2026-09-22T17:05:00+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"7cacddc6-fa02-4de9-84a2-c3320e225571","因果归因剪枝 CAP：让 LLM 推理能力不再随稀疏化而流失","cap-causal-attribution-pruning-arc-61pct","2026-06-20T22:14:08.915874+00:00"]