[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-mit-compressm-ssm-training-time-compress":3,"topics-all":36,"news-related-6df953d8-5371-47e5-94e1-2a4a0d629e4a":55},{"id":4,"title":5,"summary":6,"content":6,"original_url":7,"source_id":8,"tags":9,"translations":23,"news_slug":29,"published_at":30,"created_at":31,"modified_at":32,"is_published":33,"publish_type":34,"image_url":13,"view_count":35},"6df953d8-5371-47e5-94e1-2a4a0d629e4a","训练时压缩SSM：MIT CompreSSM如何让状态空间模型「边学边瘦」","主流模型压缩都在训练完成后做——蒸馏、量化、剪枝，全是后处理。但MIT CSAIL、Max Planck、ETH和Liquid AI联合提出了一种反直觉的方法：压缩不是训练的收尾工作，而是训练的一部分。\n\n这项技术叫CompreSSM（Compressive State Space Models），核心思路来自控制理论：用Hankel奇异值（HSV）分析状态维度，在训练过程中动态丢弃低贡献方向，让模型边学边瘦。\n\n为什么这很反常识\n\n通常来说，我们应该先让模型充分训练，再做压缩。但论文发现了一个矛盾现象：直接训练一个小模型，收敛后的性能往往不如先训大模型再压缩到同等规模。原因是训练过程中，模型会探索很多参数方向，其中一部分在训练中期有效，但在后期成为冗余——而这些冗余在传统流程里只能白白占用计算资源。\n\nCompreSSM在训练的前10%步骤内插入平衡截断（balanced truncation）：计算每个状态维度的HSV，保留高贡献方向，丢弃低能量方向。由于截断后的系统继承H∞误差边界，压缩不会导致性能崩溃，反而因为减少了优化空间的维度，让收敛更高效。\n\nSSM：长上下文的新选择\n\n状态空间模型（SSM）近年来成为Transformer的有力竞争者。Mamba等架构将序列处理复杂度从O(n²)降至O(n)，在长上下文任务上优势明显。但SSM的状态维度本身仍然是瓶颈——高维状态意味着更高的存储和更新成本。CompreSSM直接压缩状态维度，比单纯架构搜索更根本。\n\n在实验中，CompreSSM在多种序列建模任务上实现了更低的内存占用和更快的推理速度，同时保留了原始模型的表达力。对于需要部署在端侧或边缘设备的场景，这种训练时压缩的方法比后训练量化更能保证模型质量。\n\n和蒸馏、量化有什么区别\n\n模型压缩的主流路线有三条：蒸馏让学生学习教师的软输出，量化将权重精度从FP16压到INT8\u002FFP8，剪枝在训练后移除冗余权重。CompreSSM的创新在于它是结构感知的——它不是移除独立的权重参数，而是系统性移除整个状态维度，保留了模型的动态系统结构，这让压缩后的SSM仍然具有理论一致性和可解释性。\n\n前瞻\n\n目前CompreSSM的验证主要在标准序列任务上，LLM级别的应用还未披露。但方向是对的：当整个行业在为推理成本头疼的时候，从训练机制本身找效率空间，比在模型训完后再打补丁更干净。如果SSM+训练时压缩能进入主流训练框架，长上下文模型的效率曲线可能会被重新改写。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2510.02823","4613a0c2-8d14-4485-b855-f8fad33c4527",[10,14,17,20],{"id":11,"name":12,"slug":12,"description":13,"color":13},"40269b40-7942-4650-9672-ed2e6524d37a","ai-technology",null,{"id":15,"name":16,"slug":16,"description":13,"color":13},"2d9c2fb0-2be5-4ad1-aedb-e9747addf355","compression",{"id":18,"name":19,"slug":19,"description":13,"color":13},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",{"id":21,"name":22,"slug":22,"description":13,"color":13},"01598627-1ea6-4b27-a5d8-874971571a71","llm",[24],{"id":25,"lang":26,"title":27,"summary":28,"content":13},"e7da97e3-e977-4a8c-bc4c-e68cd123535c","en","CompreSSM: MIT compresses state-space models while training","MIT's CompreSSM paper proposes training-time compression for state space models, integrating compression objectives directly into the training loop. The result is SSMs that are significantly smaller than the baseline while maintaining quality, with the compression being \"learned\" rather than applied post-hoc.","mit-compressm-ssm-training-time-compress","2026-06-02T13:15:00Z","2026-06-02T13:11:10.535742Z","2026-08-19T02:08:40.142862Z",true,"agent",138,[37,46],{"slug":38,"tag_slug":38,"title_zh":39,"title_en":40,"intro_zh":41,"intro_en":42,"id":43,"is_active":33,"created_at":44,"modified_at":45},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":47,"tag_slug":47,"title_zh":48,"title_en":49,"intro_zh":50,"intro_en":51,"id":52,"is_active":33,"created_at":53,"modified_at":54},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":56},[57,62,67,72,77,82],{"id":58,"title":59,"news_slug":60,"published_at":61},"b2625716-65d0-4ec2-a6b2-6a6addb67721","MAESTRO 把 MoE 专家剪枝扔进马尔可夫链：50% 压缩下鲁棒性维度反涨 10pp","maestro-moe-expert-pruning-markov","2026-07-09T15:32:54+00:00",{"id":63,"title":64,"news_slug":65,"published_at":66},"28c41f06-d20f-481c-b133-cd109af3aed1","答对之后停不下来:微软团队揪出在线蒸馏的 EOS 错配元凶","eos-mismatch-opd-length-inflation","2026-09-18T21:09:06+00:00",{"id":68,"title":69,"news_slug":70,"published_at":71},"e91b3add-4f1d-48d3-ab7a-bd2c6e8c1765","QuIP 崩、OPTQ 降级:Kashin-DCT 在 4-bit 量化压力测试里活了下来","kashin-dct-2bit-llm-quantization","2026-09-12T15:10:00+00:00",{"id":73,"title":74,"news_slug":75,"published_at":76},"c83af54b-79ed-445c-9482-07d98c26c36b","BeaconKV:长推理会回头看,只压最近窗口的 KV 缓存注定丢东西","beaconkv-beacon-query-kv-cache-compression","2026-09-09T11:25:00+00:00",{"id":78,"title":79,"news_slug":80,"published_at":81},"9825e20d-c9eb-4300-99b5-12eb7d0e755d","自信的错误教师最危险:TGOPD 给在线蒸馏装提示级门控,教师 GPU 利用率 9.8% 升至 78.9%","tgopd-teacher-gated-on-policy-distillation","2026-09-08T23:10:00+00:00",{"id":83,"title":84,"news_slug":85,"published_at":86},"58ed753e-ad6d-4aac-95f4-36bf217e169c","把 10 万条人类视频变成机器人教材:RoboTok 检索 mAP 提升约 50 倍,hard 任务 79.3% 对 19.5%","robotok-retrieval-benchmark-reread","2026-09-06T21:11:25+00:00"]