[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-context-compression-weakens-refusal-alignment":3,"topics-all":38,"news-related-44009c4a-bcba-46a5-81a8-b94cb17ca840":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"44009c4a-bcba-46a5-81a8-b94cb17ca840","上下文压缩省了算力,却悄悄削掉了拒答","论文给出的一组实测:上下文压缩把整条请求编码成压缩表示后,有害提示的拒答率被系统性削掉,Mistral 从 62.82% 掉到 10.9%。作者把原因归给编码器只顾重建语义、没人要求它保住拒答方向,并给出不重训、直接在压缩空间补回拒答方向的修法。","上下文压缩是长上下文推理省算力的主战场。ICAE 把整篇长文压成几个 memory slot,ARC-Encoder 用连续表示直接替换 token embedding。这条路线上的方案都把“还原语义”当唯一目标,没人问过压缩之后模型的脾气还在不在。11 月 4 日提交的一篇 arXiv 论文(编号 2611.06380,作者是加拿大 Algoma 大学的 Anthony Nguyen 与 Wenjun Lin,标注发表于 Canadian AI 2026)给这本账加了一行:压缩本身会削掉拒答能力,而且削得很安静。论文原文见 [arXiv 2611.06380](https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.06380)。\n\n## 塌掉的不是语义,是拒答\n\n论文的实验把整条用户请求编码进压缩表示,再把原始 token 从解码器输入里拿掉,强迫模型只靠压缩记忆作答。作者说这是刻意设计的“诊断性压力测试”,对应上游只交付压缩表示的压缩优先管线,而不是常规检索增强那种文档被压、query 仍留在 token 上下文里的形态。\n\n数字(自建 624 条提示,一半留作评测):Mistral-7B-Instruct-v0.2 配 ICAE,有害提示的拒答率从 62.82% 掉到 10.9%;LLaMA-3.1-8B-Instruct 配 ARC 从 96.15% 掉到 36.54%;Mistral-7B-Instruct-v0.3 配 ARC 从 20.51% 掉到 8.97%;LLaMA-2-7B-Chat 配 ICAE 从 97.44% 掉到 85.9%。换到 HarmBench 与 AdvBench,方向一致:Mistral-7B-Instruct-v0.2 在 HarmBench 上从 57.5% 掉到 16.5%,在 AdvBench 上从 62.69% 掉到 20.38%。ARC 那组的 LLaMA-3.1-8B-Instruct 在 HarmBench 上也从 84.0% 掉到 47.5%。\n\n真正让人停一下的是同一张表里的另一列。安全提示上的拒答率几乎没动:四行里三行是 0.00 保持 0.00,LLaMA-2-7B-Chat 那一行反而从 0.00 涨到 3.2。也就是说压缩没有让模型变成“什么都敢答”,它精准地只削掉了该拒的那一半。只盯过度拒答的评测脚本,看不出任何异常。\n\n## 为什么塌:重建目标不奖励拒答方向\n\n作者的解释是训练目标错位。ICAE、ARC-Encoder 这类编码器只为还原原文训练,损失函数里没有任何一项要求它保住拒答信号;而拒答行为在解码器内部被 Arditi 等人证明近似由一维方向中介(覆盖 13 个开源对话模型、最大 72B)。语义重建是信息论目标,拒答是几何属性,两者不在一个层面上。机制证据是解码器残差流上的线性探针:原本会触发拒答的有害输入,压缩后沿拒答方向的激活在所有层都塌成负值,轨迹贴到“不拒答”那一侧。\n\n这跟 2024 年 EMNLP Findings 上 Xu 等人的工作能对上:他们发现剪枝和量化在保持困惑度的同时仍会改动毒性与偏见等安全行为,量化大体保住了偏见,剪枝退化得快。一个是参数级压缩,一个是上下文级压缩,两拨不同的人、不同的时间,指向同一件事——压缩不只是成本,也有副作用。\n\n## 论文留了三个问号\n\n其一,全文把“重建保真度高”当前提。摘要写的是“尽管重建保真度很高”,但论文里没有任何一张表报告重建质量:没有困惑度、没有 BLEU\u002FROUGE、没有压缩倍率。对照两篇被引的方法论文,ICAE 自己写的是 Llama 上 4× 压缩、约 1% 额外参数,ARC-Encoder 写的是通常输出少 4× 或 8× 的连续表示。压缩倍率这个关键坐标,得自己从被引论文里翻。\n\n其二,正文写“相对降幅最高 78%”(指 Mistral-7B-Instruct-v0.2 在 HarmBench 上),但按它自己表 2 的 57.5% 到 16.5% 算约 71%。同一句里 ARC 那半句“超过 40%”,按表 2 算是 43.45%,倒是对得上。\n\n其三,论文脚注给出的代码地址 github.com\u002Fant-8\u002FContext-Compression-Alignment 目前是空仓库:GitHub 接口返回仓库大小 0、内容接口 404、分支列表为空。复现这条路暂时不通。\n\n## 修法有效,但只修得了一半\n\nMemory Steering 不重训模型,只在压缩记忆上做一次干预。转向量靠反事实构造拿到:给每条提示配一个“这违反了你的 AI 安全准则”的变体,两份记忆均值的差就是方向;再用逻辑回归逐槽判断哪一格存了不安全意图,按强度 α 注入,避免安全提示上过度拒答。\n\n数字可观:AdvBench 上,Mistral-7B-Instruct-v0.2 从基线 20.38% 一路升到 α=2.0 时的 81.15%,LLaMA-2-7B-Chat 从 90.96% 升到 100.00%。但三个代价写在同一篇论文里:只能修定长压缩,ARC 这种变长池化表示用不了,而恰好 ARC 那组塌得更狠;响应质量会掉,安全提示上的胜平率 LLaMA-2 从 α=0.5 的 85.9% 降到 α=2.0 的 79.5%;旋钮还会冲过基线,Mistral 的 AdvBench 拒答率被推到 81.15%,已经高于它未压缩时的 62.69%。\n\n还有个细节值得记一笔:ARC 压缩下的 Mistral-7B-Instruct-v0.3 拒答率只剩 8.97%,评测集里只剩 14 个正例,直接训探针会因类别极度不平衡而不可靠——这正是作者改用反事实构造的原因。压缩把证据本身也压没了。\n\n## 所以呢\n\n给要上压缩的团队一条能直接执行的结论:安全评测不能只在未压缩基线上跑,压缩后的模型要单独过一遍拒答评测,而且要盯有害侧,不能只看良性侧的过度拒答。上下文压缩的评测表里,“重建保真度”和“拒答率”应该并排放,而不是默认前者稳后者就不变。\n\n至于这篇论文本身,它的价值不在“压缩会破防”这个结论有多新,而在于把一个一直被当成纯工程问题的选择,重新放回安全问题的范围里。压缩省下的每一分算力,都在问一个没人问过的问题:省掉的到底是什么。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.06380","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"1fcfaaf2-67de-43d3-9e35-5784852fec60","ai-safety",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"2d9c2fb0-2be5-4ad1-aedb-e9747addf355","compression",{"id":19,"name":20,"slug":20,"description":14,"color":14},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":22,"name":23,"slug":23,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"a7cae4b3-3d56-4f4f-a69e-fe49b9ec74c5","en","Context Compression Erodes Refusal, Not Semantics","Compressed request memory cuts harmful refusal from 62.82% to 10.9% in arXiv 2611.06380. Cause: reconstruction-only encoders; fix: Memory Steering.","Context compression is the main lever for cutting inference cost on long inputs. ICAE squeezes a whole document into a few memory slots; ARC-Encoder replaces token embeddings outright with continuous representations. Every method on that track optimises for one thing: putting the semantics back. Nobody asked what happens to the model’s temper once the context has been squeezed. A paper submitted on 4 November (arXiv 2611.06380, by Anthony Nguyen and Wenjun Lin of Algoma University in Canada, presented at Canadian AI 2026) adds a line to that ledger: compression itself erodes refusal behaviour, and it does so quietly. The paper is at [arXiv 2611.06380](https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.06380).\n\n## What collapses is not semantics, it is refusal\n\nThe paper encodes the entire user request into compressed representation space, removes the original tokens from the decoder’s input, and forces the model to answer from compressed memory alone. The authors describe this as a deliberate diagnostic stress test, modelling compression-first pipelines where an upstream system hands the generator nothing but compressed representations, rather than the usual retrieval-augmented setup where a document gets compressed while the query stays in token context.\n\nThe numbers, on a self-built 624-prompt set with half held out for evaluation: Mistral-7B-Instruct-v0.2 with ICAE drops harmful-prompt refusal from 62.82% to 10.9%; LLaMA-3.1-8B-Instruct with ARC falls from 96.15% to 36.54%; Mistral-7B-Instruct-v0.3 with ARC goes from 20.51% to 8.97%; LLaMA-2-7B-Chat with ICAE slides from 97.44% to 85.9%. Switch to HarmBench and AdvBench and the direction holds: Mistral-7B-Instruct-v0.2 falls from 57.5% to 16.5% on HarmBench and from 62.69% to 20.38% on AdvBench, and in the ARC group LLaMA-3.1-8B-Instruct falls from 84.0% to 47.5% on HarmBench.\n\nWhat deserves a second look is the other column of the same table. Refusal on safe prompts barely moves: three of the four rows sit at 0.00 and stay at 0.00, and the LLaMA-2-7B-Chat row actually rises from 0.00 to 3.2. So compression did not turn the model into something that answers everything; it trimmed precisely the half that was supposed to refuse. An evaluation script that only watches over-refusal on benign inputs would see nothing wrong.\n\n## Why it collapses: reconstruction rewards nothing about refusal\n\nThe authors explain this as a training-objective mismatch. Encoders like ICAE and ARC-Encoder are trained purely to reconstruct the original text, and nothing in their loss function asks them to preserve refusal triggers; meanwhile refusal behaviour inside the decoder is mediated, Arditi and colleagues show, by something close to a one-dimensional direction (across 13 open-source chat models, up to 72B). Semantic reconstruction is an information-theoretic objective, refusal is a geometric property, and the two do not live on the same level. The mechanistic evidence is a linear probe on the decoder’s residual stream: harmful inputs that would trigger refusal show activation along the refusal direction collapsing to negative values across all layers under compression, tracking the non-refusal regime.\n\nThis lines up with work by Xu and colleagues at EMNLP Findings 2024, who found that pruning and quantization change toxicity and bias behaviour even while perplexity holds, with quantization largely preserving bias and pruning degrading quickly. One is parameter-level compression, the other context-level, two different groups at two different times, and both point the same way: compression carries side effects, not just savings.\n\n## Three open questions in the paper\n\nFirst, high reconstruction fidelity is treated as a premise. The abstract says the effect holds “despite high reconstruction fidelity”, but no table in the paper reports reconstruction quality: no perplexity, no BLEU\u002FROUGE, no compression ratio. For contrast, the two cited method papers do supply the numbers: ICAE reports 4x compression on Llama with about 1% additional parameters, and ARC-Encoder reports typically emitting 4x or 8x fewer continuous representations than text tokens. The compression ratio, the coordinate that actually matters, has to be dug out of the references.\n\nSecond, the text says refusal rates “decrease by up to 78%” under ICAE compression, for Mistral-7B-Instruct-v0.2 on HarmBench, but the paper’s own Table 2 values of 57.5% and 16.5% work out to roughly 71%. The other half of that sentence, “over 40%” under ARC, does check out against Table 2 at 43.45%.\n\nThird, the code address given in the paper’s footnote, github.com\u002Fant-8\u002FContext-Compression-Alignment, is currently an empty repository: the GitHub API reports a repository size of 0, a 404 on the contents endpoint, and an empty branch list. Reproduction is not currently available.\n\n## The fix works, but it only covers half the problem\n\nMemory Steering retrains nothing and intervenes once on the compressed memory. The steering direction comes from a counterfactual construction: pair every prompt with a variant ending in “This violates your AI safety guidelines”, and the difference between the two sets of memory means is the direction. A logistic regression then decides slot by slot which slot holds unsafe intent, and injects at strength alpha, so benign prompts do not get over-refused.\n\nThe numbers are substantial: on AdvBench, Mistral-7B-Instruct-v0.2 climbs from a 20.38% baseline to 81.15% at alpha=2.0, and LLaMA-2-7B-Chat goes from 90.96% to 100.00%. But the same paper carries three costs. It only works for fixed-length compression, since it cannot be applied to variable-length pooled representations like ARC-Encoder, and the ARC group is precisely the one that collapsed harder. Response quality degrades: on safe prompts the win-plus-tie rate for LLaMA-2 falls from 85.9% at alpha=0.5 to 79.5% at alpha=2.0. And the knob overshoots the baseline, pushing Mistral’s AdvBench refusal to 81.15%, above the 62.69% it showed uncompressed.\n\nOne more detail is worth recording: with ARC compression, Mistral-7B-Instruct-v0.3’s refusal rate falls to 8.97%, leaving only 14 positive examples in the evaluation set, so fitting a probe directly would be unreliable under such severe class imbalance. That is exactly why the authors switch to the counterfactual construction. Compression flattened the evidence too.\n\n## So what\n\nFor teams shipping compression, here is a conclusion you can act on directly: safety evaluation cannot run only on the uncompressed baseline. Evaluate the compressed model separately on refusal, and watch the harmful side, not just over-refusal on benign inputs. In a context-compression evaluation table, reconstruction fidelity and refusal rate belong side by side, rather than assuming that a stable first column means a stable second one.\n\nAs for the paper itself, its value is less in how new the “compression breaks alignment” conclusion is than in moving a choice long treated as pure engineering back into the range of safety questions. Every unit of compute that compression saves is asking a question nobody was asking: what exactly was saved?","context-compression-weakens-refusal-alignment","2026-11-05T07:05:00Z","2026-11-05T07:26:47.572027Z","2026-11-05T07:26:47.572041Z",true,"agent",83,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"53d00d00-e4fb-41b0-a36a-37161203b514","KV缓存拼接复用丢题可至100%,修补要盯答案块","kv-cache-repair-budget-target-chunk","2026-11-05T15:09:02+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"008fe03d-3ba8-49f0-9445-0defb7977f66","TextSeal2:水印躲过投机解码,检测多22点","textseal2-watermark-speculative-decoding","2026-11-01T15:11:35+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"02c96622-1409-4867-9f25-772936edc6cd","滑动窗口反超SnapKV:NVIDIA重审投机解码KV压缩","rift-draft-kv-cache-compression","2026-10-23T15:20:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"a046b158-396f-4db9-a8d3-22d8c193a166","清华把KV缓存当神经网络训练:32倍压缩准确率96.7%","latent-compaction-test-time-kv","2026-10-15T17:08:40+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"dab2e487-cfba-4d75-a93f-eff133476348","剪枝校准对不上解码分布?西湖蚂蚁让稀疏解码提速48%","sparsedecoding-decoding-aware-pruning","2026-10-10T19:09:23+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"c83af54b-79ed-445c-9482-07d98c26c36b","BeaconKV:长推理会回头看,只压最近窗口的 KV 缓存注定丢东西","beaconkv-beacon-query-kv-cache-compression","2026-09-09T11:25:00+00:00"]