[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-caplite-slm-prompt-injection-defense":3,"topics-all":38,"news-related-bcc44cdc-2997-42e6-a4dc-0823642c9c9f":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"bcc44cdc-2997-42e6-a4dc-0823642c9c9f","小模型写不好自己的防线：14\u002F15模型被攻破","执行型防注入方案常让 agent 自己的模型写安全工件。论文实测 15 个开源模型：模型写的权限策略在 14 个上放行攻击 676 次，CaMeL 程序在个别模型上 68% 跑不起来。CapLite 从工具 schema 派生策略、先提交写再读，13,935 次攻击零得手，token 只要 CaMeL 的 18%。","Agent 安全圈有一条常见的分工：模型负责干活，代码负责把关。CaMeL、Progent、IPIGuard、DRIFT 这批「执行型」防注入方案都是这么设计的——但它们交给代码去执行的那个安全工件，也就是程序、权限策略、依赖图或约束清单，依然让 agent 自己的模型来写。arXiv 新论文（2611.17115）的三位作者 Zimo Xu、Wenzhuo Liu 和 Hideki Nakayama 把这个问题摆上台面：小模型，写得动自己的防线吗？\n\n## 实验怎么做的\n\n测试床是 AgentDojo v1.2.2 的四个套件（Workspace、Slack、Banking、Travel），97 个用户任务，949 个任务-注入配对，攻击成功率按其中 929 个需要工具调用的配对计算。模型侧一共 15 个开源权重模型：Qwen3-1.7B\u002F-4B\u002F-8B、Qwen3.5-4B\u002F-9B、Llama-3.1-8B-Instruct、Gemma-4-E2B\u002F-E4B\u002F-12B、Granite-4.2-8B、Ministral-3-8B、LFM2.5-2.6B 等 13 个小模型，再加 Qwen3-32B 和 MoE 架构的 Qwen3.6-35B-A3B 两个参照。主攻击是 AgentDojo 自带的 important_instructions，稳健性实验另加点名工具的 tool_knowledge，以及两个专门冲着 CapLite 设计的变体。\n\n## 写「允许什么」的工件，朝失败敞开\n\nProgent 让模型从请求里写权限策略、每次调用后再更新。结果是 15 个模型里 14 个的策略放行了攻击，累计 676 次，Qwen3-32B 单模型被攻 132\u002F929。同一批策略还会挡掉 25–77% 的正常写调用，良性效用从 59.4% 掉到 43.8%——同一个工件，既漏攻击又误伤自己人。IPIGuard 的依赖图在 13\u002F15 个模型上被破（435 次），DRIFT 的轨迹清单在 11\u002F15 个上被破（214 次）；DRIFT 那四个零失守的模型，攻击下任务完成率只有 4.7%–9.5%，零来自不作为。\n\n## 写「运行什么」的工件，朝失败关死\n\nCaMeL 走另一头：解释器强制执行数据流策略，安全性确实成立——13,935 个受攻击回合只失守 20 次，其中 17 次集中在 LFM2.5-2.6B。代价是 15 个模型里 8 个在超过 15% 的回合产不出能运行的程序，Granite-4.2-8B 上达到 68%，模型干脆不返回代码；格式、语法和解释器错误能扛过十轮修复，每回合烧到 94k token，良性效用掉到 24.9%。\n\n## 更大不等于更稳\n\nQwen3 从 1.7B 换到 32B，Progent 下的攻击成功率从 3.0% 涨到 14.2%；Gemma-4 从 0% 涨到 12.7%。模型越大越好注入，而它写的工件并没有把多出来的风险吸收掉。跨 15 个模型看，工件失败率与模型能力没有显著相关（相关系数绝对值不超过 0.33，p 值大于 0.2）。\n\n## CapLite：模型提案，代码授权\n\nCapLite 的答案是干脆不让模型碰授权。策略从工具 schema 按固定规则派生，允许的值只取用户请求里明说的。Planner 只看请求就把写调用提交，之后才允许读任何工具结果；Mediator 只放行每一个决定影响对象（收件人、金额、对象标识、URL）都等于请求原文给出值的写调用。于是 13,935 个受攻击回合零得手——包括诱导读取攻击者 URL 的 read exfiltration（0\u002F168）和伪造用户描述符的 descriptor collision（0\u002F104）这两个为它量身定制的攻击。受攻击效用与 CaMeL 相当（29.0% 对 27.2%），每回合 17.1k token，是 CaMeL 93.7k 的 18%，比不设防 agent 的 28.0k 还低；按论文的 utility-per-thousand-tokens 口径为 1.69，对 DRIFT 的 1.24 和 CaMeL 的 0.29。\n\n## 论文自己承认的账单\n\n所有结果都出自 AgentDojo 四个套件这一个基准；受攻击效用比不设防低 21 个点（29.0% 对 50.2%），43% 的良性失败是提交的计划漏掉了任务需要的写调用，38% 是写都执行了任务仍失败——没有代码托底的计划没法循环和分支；在 Gemma-4-E4B 和 -12B 上效用显著低于 CaMeL（这两款写 CaMeL 程序可靠，失败率仅 3–14%）；请求里没明说的授权值，只能回头向用户确认。\n\n论文结论里那句话比 0\u002F13,935 更值得记住：小模型不能被信任去写自己的防线——它们也不需要。把授权交给确定性代码之后，真正的问题变成：你的 agent 栈里，还有多少安全决策，其实仍然是模型在拍脑袋？\n\n论文：https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.17115","https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.17115","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"6ad31a14-c0da-42df-81fd-564281f768db","agentic-ai",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"1fcfaaf2-67de-43d3-9e35-5784852fec60","ai-safety",{"id":19,"name":20,"slug":20,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":22,"name":23,"slug":23,"description":14,"color":14},"02bfdc0f-5e23-4fbb-b31a-7cc03cdba1ee","security",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"e22acd29-ac0f-4b90-b912-58e823157a95","en","CapLite: Small Models Cannot Write Their Own Defenses","Model-written defenses fail on small models: 676 attacks admitted, CaMeL programs unrunnable in 68% of episodes; CapLite stops all 13,935 attacks.","Enforcement defenses against prompt injection like CaMeL, Progent, IPIGuard and DRIFT share a design: take the security decision out of the model's context and give it to code. But the artifact that the code enforces — a program, a privilege policy, a dependency graph or a constraint list — is still written by the agent's own model. A paper posted to arXiv (2611.17115) by Zimo Xu, Wenzhuo Liu and Hideki Nakayama asks a blunt question: can small language models write these artifacts at all?\n\n## The setup\n\nThe testbed is AgentDojo v1.2.2 with its four suites (Workspace, Slack, Banking, Travel): 97 user tasks and 949 task-injection pairs, with attack success measured on the 929 pairs whose injection goal needs a tool call. Fifteen open-weight models: Qwen3-1.7B\u002F-4B\u002F-8B, Qwen3.5-4B\u002F-9B, Llama-3.1-8B-Instruct, Gemma-4-E2B\u002F-E4B\u002F-12B, Granite-4.2-8B, Ministral-3-8B and LFM2.5-2.6B among thirteen small models, plus two larger references, Qwen3-32B and the mixture-of-experts Qwen3.6-35B-A3B. The main attack is AgentDojo's important_instructions; robustness runs add tool_knowledge and two attacks written specifically against CapLite's design.\n\n## Artifacts that decide what is allowed fail open\n\nProgent has the model write a privilege policy from the request and update it after each call. Those policies admit attacks on 14 of 15 models, 676 in total, with Qwen3-32B alone at 132 of 929. The same policies block 25-77% of the write calls the agent attempts under attack, and benign utility drops from 59.4% to 43.8%: one artifact, wrong in both directions. IPIGuard's dependency graphs are breached on 13 of 15 models (435 attacks), and DRIFT's trajectory checklists on 11 of 15 (214 attacks). DRIFT's four clean models complete only 4.7-9.5% of attacked tasks — their zero comes from inaction.\n\n## Artifacts that decide what runs fail closed\n\nCaMeL goes the other way: its interpreter enforces data-flow policies on whatever program the model writes, and security holds — 20 successful attacks in 13,935 episodes, 17 of them on LFM2.5-2.6B. The price: on 8 of the 15 models the final program is unrunnable in more than 15% of episodes, reaching 68% on Granite-4.2-8B, where the model returns no code at all. Format, syntax and interpreter errors survive ten repair rounds, the bill reaches 94k tokens per episode, and benign utility falls to 24.9%.\n\n## Bigger is not safer\n\nMoving up the Qwen3 family from 1.7B to 32B raises the Progent attack success rate from 3.0% to 14.2%; on Gemma-4 it goes from 0% to 12.7%. Larger models are easier to inject, and the artifacts they write do not absorb the extra risk. Across all 15 models, no artifact failure rate shows a significant trend with capability (correlation magnitude at most 0.33, p above 0.2).\n\n## CapLite: the model proposes, code authorizes\n\nCapLite's answer is to never let the model decide what is authorized. The policy is derived from the tool schemas by fixed rules, and the allowed values are exactly what the user's request states. A Planner that sees only the request commits the write calls before any tool result is read; a Mediator admits a write only when every authority argument — recipient, amount, object identifier, URL — equals a value stated in the request. No attack succeeds in any of the 13,935 attacked episodes, including the read exfiltration attack built to defeat it (0\u002F168) and the descriptor collision attack (0\u002F104). Utility under attack is comparable to CaMeL's (29.0% vs 27.2%) at 17.1k tokens per episode — 18% of CaMeL's 93.7k and below the undefended agent's 28.0k — with 1.69 utility per thousand tokens against 1.24 for DRIFT and 0.29 for CaMeL.\n\n## The bill the authors admit\n\nEverything comes from one benchmark, AgentDojo's four suites. Utility under attack is 21 points below the undefended agent's (29.0% vs 50.2%): in 43% of CapLite's benign failures the committed plan misses a write the task needs, and in 38% every committed write runs but the task still fails — a plan without code cannot loop or branch. On Gemma-4-E4B and -12B utility is significantly below CaMeL's (those two write CaMeL programs reliably, failing only 3-14% of the time). Authority values the user never stated require a confirmation round-trip.\n\nThe line worth remembering is not the 0\u002F13,935 but the conclusion: small models cannot be trusted to write their own defenses — and they do not need to. Once authorization belongs to deterministic code, the question becomes: how many security decisions in your agent stack are still a model improvising?\n\nPaper: https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.17115","caplite-slm-prompt-injection-defense","2026-11-14T13:09:01Z","2026-11-14T13:09:01.463857Z","2026-11-14T13:09:01.463873Z",true,"agent",96,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"92267597-eb39-4323-a5db-f7db3821f392","安全对齐被反用:一条假差评就能改掉智能体的选择","voluntary-servitude-agent-safety-attack","2026-11-05T01:05:00+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"19dd65ff-646c-4edf-bd90-98b2dc8bf215","Web Agent幻觉可解释性:探针能预警,归因却失灵","web-agent-hallucination-mechanistic-interpretability-probe","2026-11-17T03:15:00+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"4fcd63ae-2b7c-43a2-9716-d7c20ca4cfd7","模型指纹会传染:中途换人,检测率0.97跌到0.72","contagious-llm-fingerprints-relay-audit","2026-11-16T13:30:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"f9174923-987b-4db8-a33c-d4c66e0498d7","水印不止辨真假:200 token 短文藏 24 比特溯源","codebook-watermark-short-text-attribution","2026-11-15T13:30:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"9649aae9-2aef-4668-96e8-d8ced4378a71","专家头衔一贴，模型就多改口：Claude掉16.1点","llm-expert-disagreement-revision-selectivity","2026-11-15T07:10:00+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"af484c32-d6f7-406e-86a0-09dc2cdabbc0","零宽字符藏在回复和邮件里:模型偷偷记下你的性别年龄","qwen-llama-synthesis-smuggling-privacy-leak","2026-11-14T03:05:00+00:00"]