[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-voluntary-servitude-agent-safety-attack":3,"topics-all":38,"news-related-92267597-eb39-4323-a5db-f7db3821f392":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"92267597-eb39-4323-a5db-f7db3821f392","安全对齐被反用:一条假差评就能改掉智能体的选择","Technion 团队在 arXiv 放出论文《Voluntary Servitude》,该论文标注被 NeurIPS 2026 主会接收。攻击者不必写指令,一条编造的差评指控就能让智能体用自己的避险逻辑淘汰正当商家。论文测得无过滤时成功率 60.4%,经过商用检测器后反而比指令注入强 1.7 到 4.4 倍。","攻击者其实不需要在网页里写命令。\n\n11 月 3 日,Technion 团队在 arXiv 放出论文《Voluntary Servitude》(自愿的奴役),标注被 NeurIPS 2026 主会接收。它研究的是 AI 智能体最公共的入口:读评论页、论坛帖、榜单页,再替用户做推荐、排序和下单建议。[论文原文](https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.04393)\n\n围绕\"间接提示注入\"的攻防建立在一个共同假设上:攻击者必须在抓到的内容里写下命令,防御方的检测器去筛命令句式。这篇论文说,这个假设反而替攻击者清出了路。\n\n## 载荷里没有一句命令\n\n攻击者做的事很朴素:在目标品类的评论页、论坛和列表页上发几条普通评论,带用户名、日期、标题,结构跟周围真实评论一模一样。特殊的是里面那句断言——比如\"这家在一场医疗急救里抛下了一家人\"。整个载荷没有一句是对模型说的。\n\n论文把攻击分成三族:安全族(编造危险,靠避险倾向)、权威族(伪造机构背书,靠对机构的顺从)、属性族(讲对方哪点更好)。安全族在无过滤时和指令注入基本打平:60.4% 对 66.0%。\n\n## 关键不是\"有指控\",是\"有危险\"\n\n论文的对照只换断言内容,位置、条数、版式都不变。换成\"这家道德败坏\"这种不涉及安全的指控,成功率 8.2%;换成纯粹好评、只推攻击者的目标,6.8%,两者统计上分不出来(p=0.14);不投毒时,目标是 0.7%。\n\n\"负面\"本身不推动智能体,\"危险\"才推动。安全族比指控族高出 22.8 到 84.4 个百分点,在十个品类、四个模型上都成立。触发点不在指控,在避险。\n\n## 检测器为什么放过它\n\n论文在两个商用检测器上测了通过率。两者会标出 46% 到 84% 的指令注入载荷,但对安全族最多只标出 22%;Lakera Guard 更极端,安全族 78% 被判为良性。真正像攻击的，是指令;而这条载荷读起来像旅行者间的证词。\n\n把通过率乘成功率,得到过滤后的有效成功率:ProtectAI 下安全族 60.4 对指令注入 35.7,Lakera 下 47.1 对 10.6。在部署了检测器的系统里,这条绕开指令的攻击比大家一直在防的那条强 1.7 到 4.4 倍。\n\n## 它靠淘汰取胜,不是靠说服\n\n智能体要么\"淘汰\"——把正当商家直接划掉,让攻击者的目标成为剩下的那个;要么\"比较\"——保留所有选项但把目标排第一。安全族在六个模型中有四个靠淘汰取胜,占成功案例的 97% 到 99%;权威族最高 64%,属性族最高 28%。在 Claude Haiku 4.5 上,成功的权威和属性攻击没有一次淘汰。\n\n模型顺序也反直觉:最不吃普通说服的 Grok 4.3 更吃安全框架(属性族 73.6%、安全族 53.7%);最顽固的 Claude Haiku 4.5 是安全族 22.5%、属性族 9.3%;Gemini 3.1 Flash Lite 的安全族高达 86.5%。避险倾向强的模型不一定更抗打,只是换个入口更好进。\n\n## 已经打到真在用的产品里\n\n论文把攻击搬到了四个生产级浏览器智能体上:Edge 的 Copilot 每次都选了攻击者的目标(指令注入 20%),Perplexity 的 Comet 是 60%(指令注入一次没中),OpenAI 的 Codex 浏览器 20%(同样没中),Brave 的助手在两种攻击下都是每次命中。安全载荷在三个上优于指令注入。\n\n再往下一层是检索管道。BM25 取前三页时端到端成功率 57.0%,稠密检索取一页只有 1.0%,而把被污染的页面直接塞进上下文是 98.0%。但只要那一页真被检索到,智能体选中攻击者目标的比例稳定在 49.0% 到 66.0%。分工很清楚:检索器决定污染页面多久露一次面,模型决定看到后怎么反应。\n\n另两项独立研究给出同一条路的背景:一项扫描 1.2B 个 URL、24.8M 台主机,在 11.7K 个页面上找到有效的间接注入,约 70% 藏在机器才读到的标记里;另一项发现,给一个被频繁检索的用户生成页面追加 13 个词,就能让攻击者的内容出现在 38% 到 51% 的报告里。\n\n## 修法不是加过滤器,是改证据标准\n\n论文给的防御是一条系统提示里的额外要求:安全相关的断言如果只出现在少数来源、或提出它的账号在措辞和时间上高度相似,就当作未证实;未证实的断言不得单独用来淘汰选项,必须先有独立佐证。\n\n不需要重训,不需要额外组件。三个模型上的合并成功率从 68.4% 掉到 3.6%(95% 置信区间 2.2% 到 5.7%),约等于去掉 95% 的暴露面。更值得看的是它起效的位置:智能体有提示时仍然引用被植入的危险内容(98.4% 对 99.8%),但几乎不再因此淘汰商家(4.0% 对 90.9%)。它继续读,只是不再照着读。\n\n代价也被标了出来:面对知道规则的对手,攻击者可以伪造互相独立的佐证,这一项留作开放问题;真正来自单一来源的真警告也会被拖住。\n\n## 所以呢\n\n这篇论文最值得带走的不是数字,而是:每一种被对齐灌进模型的行为,都是一条会读取外部证据的决策规则,而外部证据不可信。避险、顺从机构、重视他人证词,每一条都能当杠杆。所以做智能体安全,不能只盯着\"文本里有没有命令\",得更追问\"模型拿什么当证据、几个来源才算够\"。\n\n论文自己也留了口子:实验只在旅游消费服务场景、单次选型决策上做,模型是三月到五月间经 OpenRouter 调用的快照,防御部分只测了不会随机应变的攻击者。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.04393","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"6ad31a14-c0da-42df-81fd-564281f768db","agentic-ai",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"1fcfaaf2-67de-43d3-9e35-5784852fec60","ai-safety",{"id":19,"name":20,"slug":20,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":22,"name":23,"slug":23,"description":14,"color":14},"02bfdc0f-5e23-4fbb-b31a-7cc03cdba1ee","security",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"e7829c91-f6e3-4c76-8863-480436ded511","en","An Attack That Uses the Agent’s Own Safety Alignment","Technion’s NeurIPS 2026 paper needs no command. A fake hazard makes an agent drop legitimate providers. Success 60.4%, 1.7 to 4.4 times injection.","The attacker never has to write a command into the page.\n\nOn November 3, Technion released a paper titled Voluntary Servitude, a play on \"voluntary servitude,\" listed as accepted at NeurIPS 2026. It studies a routine entry point for AI agents today: reading review pages, forum threads and listing pages, then making a recommendation, a ranking or a purchase decision on the user’s behalf. [Paper](https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.04393)\n\nThe defence against indirect prompt injection rests on a shared assumption: the attacker must write a command into the retrieved content, and the deployed injection detector screens for command syntax. This paper argues that the assumption itself hands the attacker an opening.\n\n## The payload contains no command\n\nThe attacker’s move is mundane. Post a few ordinary reviews on the review pages, forums and listings for the target service category, complete with a username, a date and a title, structured exactly like the genuine reviews around them. What is unusual is the claim inside. For example: \"this provider abandoned our family during a medical emergency.\" Not one line of the payload is addressed to the model. There is no instruction, no override of the system prompt, no request to adopt a persona.\n\nThe paper splits the attack into three families. The safety family fabricates a hazard and leans on the model’s harm-avoidance. The authority family fabricates an institutional endorsement and leans on deference to institutions. The attribute family simply argues that the attacker’s listing is better and leans on ordinary persuasion. With no filter in place, the safety family matches instruction injection almost exactly: 60.4% against 66.0%.\n\n## Not an accusation, a hazard\n\nThe paper runs a clean control. Position, review count and page format are held fixed, and only the claim changes. Replace the hazard with an accusation of ethical misconduct, which carries no safety content, and success drops to 8.2%. Replace it with a purely positive review that only promotes the attacker’s target, and success is 6.8%, which the authors cannot tell apart statistically from the misconduct case (p=0.14). With no poisoning at all, the target is chosen on 0.7% of queries.\n\nNegative content alone does not move the agent. Hazard content does. The safety family beats the misconduct complaint by 22.8 to 84.4 percentage points, and it holds across all ten domains and all four models. The trigger is not the accusation. It is harm-avoidance.\n\n## Why the detectors wave it through\n\nThe paper measures pass rates on two real commercial detectors. Both flag 46% to 84% of instruction injection payloads, but flag at most 22% of the safety family. Lakera Guard is the more extreme case, scoring 78% of the safety family as benign. What looks like an attack is an instruction. This payload reads like testimony between two travellers.\n\nMultiplying the pass rate by the success rate gives the effective success rate behind the filter. Under ProtectAI the safety family reaches 60.4 against 35.7 for instruction injection; under Lakera, 47.1 against 10.6. In systems where such a filter is actually deployed, the attack that avoids instructions lands 1.7 to 4.4 times harder than the one everyone has been defending against.\n\n## It wins by elimination, not persuasion\n\nAn agent can reach the attacker’s listing in two ways. It can eliminate the legitimate providers, so that the target is what remains, or it can compare, keeping every option and ranking the target first. On four of the six models, the safety family succeeds mostly by elimination, accounting for 97% to 99% of its successful attacks. The authority family reaches 64% at most, and the attribute family 28%. On Claude Haiku 4.5, no successful authority or attribute attack eliminated a provider at all.\n\nThe model ordering runs against intuition. Grok 4.3, which resists ordinary persuasion, follows the safety framing more readily (73.6% attribute, 53.7% safety). Claude Haiku 4.5 resists both other families, showing 22.5% safety against 9.3% attribute. Gemini 3.1 Flash Lite reaches 86.5% on the safety family. A model with stronger harm-avoidance is not harder to attack; it simply offers a more convenient entrance.\n\n## It reaches products in production\n\nThe paper moves the attack into four production browser agents. Microsoft’s Edge with Copilot selected the attacker’s listing in every trial, against 20% for instruction injection. Perplexity’s Comet was at 60%, with instruction injection never succeeding there. OpenAI’s Codex browser was at 20%, also with instruction injection never succeeding. Brave’s assistant selected it under both attacks in every trial. The safety payload outperforms instruction injection on three of the four.\n\nOne layer down sits the retrieval pipeline. With BM25 selecting the top three pages, end-to-end success reaches 57.0%. With a dense retriever selecting a single page, it is 1.0%. Handing the model the poisoned page directly gives 98.0%. But whenever that page is genuinely retrieved, the agent picks the attacker’s target on 49.0% to 66.0% of queries, and that conditional rate barely moves with the retrieval setting. The division of labour is clear: the retriever decides how often a poisoned page surfaces, and the model decides what to do once it does.\n\nTwo independent studies supply background on the same road. One crawl covered 1.2B URLs across 24.8M hosts and found validated indirect injections on 11.7K pages, roughly 70% of them hidden in markup that only a machine reads. Another found that appending 13 words to a frequently retrieved user-generated page is enough for the attacker’s content to surface in 38% to 51% of generated reports.\n\n## The fix changes what counts as evidence\n\nThe proposed defence is one extra requirement in the agent’s system prompt. A safety-critical claim is treated as unverified when it appears in only a small minority of sources, or when the accounts making it look alike in wording and timing. An unverified claim may not disqualify a provider on its own; independent corroboration is required first.\n\nNo retraining and no extra infrastructure. Pooled across three models, success falls from 68.4% to 3.6% (95% CI 2.2 to 5.7%), which removes roughly 95% of the exposure. The more revealing number is where the instruction acts. With the requirement in place, the agent still cites the planted hazard in nearly every answer (98.4% against 99.8%), yet it almost never eliminates a provider on that basis (4.0% against 90.9%). It keeps reading. It simply stops obeying.\n\nThe cost is stated too. Against an attacker who knows the rule, and can manufacture apparently independent corroboration, that setting is left open. And a genuine warning from a single source now waits for corroboration as well.\n\n## So what\n\nThe point worth carrying away is not a number. Every behaviour alignment has written into a model is a decision rule that reads external evidence, and external evidence is untrusted. Harm-avoidance, deference to institutions, the weight given to other people’s testimony: each one can be used as a lever. So agent security cannot stop at asking whether the text contains a command. It has to ask what the model treats as evidence, and how many independent sources count as enough.\n\nThe paper leaves its own limits visible. The experiments cover travel and consumer services, single selection decisions, models called through OpenRouter between March and May, and a defence evaluated only against an attacker who does not adapt.","voluntary-servitude-agent-safety-attack","2026-11-05T01:05:00Z","2026-11-05T01:31:54.481886Z","2026-11-05T01:31:54.481907Z",true,"agent",143,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"9785c9bb-85a0-4b97-8890-aa2d6808bf86","Agent记忆陷阱:良性对话把不安全操作率推到81%","agent-memory-trap-benign-overgeneralization","2026-11-08T15:12:52+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"0b01985c-c1dd-45d2-ba29-2a1a9a6afa25","前沿模型断网训练:OpenAI 造了个离线世界","openai-frontier-training-offline-world","2026-10-24T05:31:47+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"c2c96e3a-cc8d-4970-8d9d-e9584bbc9237","WhatsApp 帮助页藏 102 块 LLM 暗指令","whatsapp-hidden-llm-instructions-help-pages","2026-10-19T16:00:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"6e2537e2-8be4-4d2f-860c-127a22c60e12","Anthropic 把 Claude 拉进基建与开源安全","anthropic-cyber-mission-oss-scanner","2026-10-08T09:00:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"9e209ccb-cd2c-44cd-acaf-b168454074b2","中国 AI 智能体也会撒谎:88% 轮次出现虚假陈述","china-ai-agents-lie-88pct-tender-sim","2026-10-04T04:00:00+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"4977d1f4-c8f7-480c-aaa1-ec01d69f44d8","8B 拿 SFT+GRPO 打 685B MoE:KaliBench 把 LLM 网络安全工具调用拆到命令行级","kalibench-cybersecurity-cli-runtime-rewards","2026-10-03T05:00:00+00:00"]