[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-opticalrag-pixel-space-rag-compression":3,"topics-all":38,"news-related-6186bff0-6bbb-4970-a0bc-b7bfa93a8e49":67},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"6186bff0-6bbb-4970-a0bc-b7bfa93a8e49","像素空间压缩:16个token装下RAG检索到的文档","把文档渲染成页面图，视觉编码器把一页压成100个token，再蒸馏成16个前缀token注入LLM。人大团队提出OpticalRAG：LongBench-v2上原始分35.79，比标准RAG高出4.38分，注入token少约101倍。复算发现CAP 49.68分里13.89分来自压缩倍率。","把检索回来的长文档压进上下文窗口，一直是 RAG 系统最贵的一环。文本压缩会不可逆地丢掉细节——删掉的字符没法再找回来；视觉编码路线则提供了另一种可能。11 月 10 日挂上 arXiv 的 OpticalRAG（中国人民大学信息学院，[论文编号 2611.13675](https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.13675)）给出的诊断是：卡点未必是信息密度不够，而是你选择在一维文本序列里压缩。一整页文档渲染成图片后，现代视觉编码器能用大约 100 个视觉 token 装下，语义损失却很小——这是纯文本路线够不到的压缩比。\n\n## 三个组件，总共约三千万可训练参数\n\n整套流程是「渲染 → 编码 → 检索 → 蒸馏」四步。文档按 24,000 字符切块，用确定性排版引擎渲染成 640×640 的页面图，送进冻结的 DeepEncoder（架构取自 DeepSeek-OCR 的 SAM-ViT-B 加 CLIP 组合，从其 checkpoint 加载），每页产出 100 个、维度 1024 的视觉 token；这些 token 逐个进离线 FAISS 索引，用点积取固定 top-256。\n\n两个真正需要训练的部分：一是 Encoder-Bridged Retrieval，把 LLM 最后一层的 query 隐状态投影进视觉特征空间，靠与压缩器共享的投影层完成，**不需要对比学习**；二是 Query-Driven Distillation，一个轻量交叉注意力压缩器，用 16 个可学习 query 把检索到的 256 个视觉 token 蒸馏成 16 个，再经 VisualToLLMBridge 投影回 LLM 的嵌入空间。\n\n冻结的 DeepEncoder 和 Qwen2.5-7B-Instruct 都不动，可训练的只有压缩器（14,187,520 参数）和桥接层（16,529,408 参数），合计 30,716,928。训练分两段：先用 Wikipedia 加 C4 的 3,800 条样本做文本重建预训练（学习率 1×10⁻⁴），再用 SQuAD v2、DROP、CoQA 的 11,400 条做任务微调（学习率 3×10⁻⁶）。仓库放出了推理代码与压缩器 checkpoint，但**不含训练流程**。\n\n## 49.68 分里，有 13.89 分是倍率给的\n\n论文的主指标叫 CAP（压缩感知准确率）：把任务分乘一个由 token 预算决定的倍率，倍率是 ((12000+1)\u002F(B+1))^0.05。我按论文给的公式把主表 21 个格子全算了一遍，**全部对得上**。但复算过程本身暴露了值得说的东西。\n\nLongBench-v2 上，方法的原始分 35.79，是论文这张表里七个方法中最高的，比标准 RAG 的 31.41 高 4.38 分，比闭卷基座 30.62 高 5.17 分，注入 token 从 1622.68 降到 16，少了约 101 倍。CAP 是 49.68，也确实是最高。问题是：35.79 乘以 1.3882 等于 49.68，也就是说，**13.89 分（占 28%）纯粹来自压缩倍率**，不是任务准确率的贡献。\n\nBAMBOO 上更能看出这个结构：原始分 61.00 其实**不如** RECOMP 的 75.50 和 LLMLingua-2 的 73.50，它赢的是 CAP（84.67 对 81.80）。LooGLE-v2 上原始分 23.54 确实是最高，但 xRAG 只注入 1 个 token、原始分 21.06，CAP 就能拿到 32.54，与本文方法的 32.67 只差 0.13 分——倍率对极小 token 预算的奖励，比原始分差距更抢眼。\n\n## 多给 token 反而更差\n\n消融实验最能说明压缩不是「越多越好」。按注入 token 数从少到多排，整体分依次是 31.0（1 个）、31.6（2 个）、31.4（4 个）、32.8（8 个）、**35.8（16 个）**、30.0（32 个）、34.2（64 个）——32 个 token 的成绩比 8 个还低，64 个也仍然不如 16 个。论文给的解释是查询条件化内在秩 r：压缩 token 数只需匹配 r，而 LongBench-v2 上 r ≤ 16，超过这个数并不会带来更多信息，只会让优化更难。\n\n## 不是一个人的想法\n\n「用像素替代文本」这条路线，眼下至少有三支团队在做。DeepSeek-OCR 在 2025 年 10 月就报告过：一页文字可压到约 100 个视觉 token，压缩比低于 10 倍时 OCR 精度仍有 97%，到 20 倍时约 60%；在 OmniDocBench 上用 100 个 vision token 就超过了每页 256 token 的 GOT-OCR2.0。2026 年 6 月的 PixelRAG 把网页直接当截图检索，在文本为主的 NQ、SimpleQA 上反而更准，相比文本基线最高提升 18.1%，靠降分辨率最多省 3 倍 token。8 月的 RAGOCR 用 GRPO 训一个压缩器给每段文档动态分配分辨率，比朴素 RAG 准确率高 15% 以上而输入 token 只要八分之一；它学出的压缩比例还能直接当 reranking 分数用，反超 Qwen3-Reranker-4B。\n\nOpticalRAG 属于这条线，但把压缩比推到了 16 个 token 的位置——比前两者激进得多。\n\n## 落地前该知道的三个限制\n\n论文自己列得很清楚：评测**只做了多选题**，开放式生成没有验证；固定的 16 token 前缀预算，对需要跨越多个相距很远的文档片段取证的问题不一定够用；自适应前缀预算留给了未来工作。仓库只给推理代码和 checkpoint，不含训练流程，想复现得自己补。\n\n所以这套东西现在更像一个方向证明，而不是可以直接抄进生产的东西。但它抛出的问题值得所有做上下文压缩的人想一想：我们一直在问「怎么把文本压得更小」，也许该问的是「信息能不能换个模态装」。\n","https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.13675","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":19,"name":20,"slug":20,"description":14,"color":14},"499f4b56-819d-49a3-9609-33e775143b86","multimodal",{"id":22,"name":23,"slug":23,"description":14,"color":14},"045c011e-e2bb-45ce-bdd6-0c927f8a3b87","token-efficiency",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"96ec018a-954a-499b-8e15-d71d1fd253b9","en","Pixel-Space RAG: 16 Tokens Replace 1,622 Retrieved Tokens","OpticalRAG renders documents as page images, compresses them to 16 prefix tokens, and scores 35.79 on LongBench-v2, 4.38 points above standard RAG.","Retrieval-augmented generation has always had an awkward bill to pay: the documents you retrieve are expensive to stuff into a context window, and text compression throws away detail irreversibly — a deleted character cannot be recovered. OpticalRAG, posted on arXiv on November 10 (School of Information, Renmin University of China; [arXiv:2611.13675](https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.13675)), suggests a different diagnosis. The bottleneck may not be insufficient information density, but the choice to compress inside a one-dimensional text sequence. Render a full document page as an image and a modern visual encoder holds it in roughly 100 visual tokens with little semantic loss — a compression ratio the purely textual route cannot reach.\n\n## Three components, about thirty million trainable parameters\n\nThe pipeline is render, encode, retrieve, distill. Documents are split into 24,000-character chunks, rendered by a deterministic typesetting engine into 640×640 page images, and passed through a frozen DeepEncoder (the SAM-ViT-B plus CLIP combination taken from the DeepSeek-OCR checkpoint), producing 100 visual tokens of dimension 1024 per page. Each visual token is stored individually in an offline FAISS index and retrieved by dot product with a fixed top-K of 256.\n\nOnly two pieces are trained. Encoder-Bridged Retrieval projects the LLM's last-layer query states into the visual feature space through a projection shared with the compressor, so no contrastive training is required. Query-Driven Distillation is a lightweight cross-attention compressor that condenses the 256 retrieved visual tokens into 16 using 16 learned queries, after which a VisualToLLMBridge projects them back into the LLM's embedding space.\n\nThe DeepEncoder and Qwen2.5-7B-Instruct stay frozen. Only the compressor (14,187,520 parameters) and the bridge (16,529,408 parameters) are trained, 30,716,928 in total. Training runs in two stages: text reconstruction pretraining on 3,800 Wikipedia and C4 samples at a learning rate of 1×10⁻⁴, then task-aware fine-tuning on 11,400 examples from SQuAD v2, DROP, and CoQA at 3×10⁻⁶. The repository ships inference code and the compressor checkpoint, but not the training workflow.\n\n## Of 49.68 points, 13.89 come from the multiplier\n\nThe headline metric is CAP (compression-aware accuracy): the task score multiplied by a budget-dependent factor, m(B) = ((12000+1)\u002F(B+1))^0.05. Recomputing all 21 cells of the main table from that formula reproduces the published numbers exactly. But the recomputation surfaces something worth reporting.\n\nOn LongBench-v2 the raw score is 35.79, the highest of the seven methods in the paper's table — 4.38 points above standard RAG's 31.41 and 5.17 points above the closed-book backbone's 30.62 — while injected tokens fall from 1622.68 to 16, roughly 101 times fewer. CAP reaches 49.68, also the highest. The catch: 35.79 × 1.3882 = 49.68, so 13.89 points, 28% of the reported CAP, are pure compression credit rather than task accuracy.\n\nBAMBOO makes the structure plainer. The raw score of 61.00 actually trails RECOMP's 75.50 and LLMLingua-2's 73.50; what wins is CAP, 84.67 against 81.80. On LooGLE-v2 the raw 23.54 is genuinely highest, yet xRAG injecting a single token with a raw score of 21.06 still lands a CAP of 32.54, just 0.13 behind the paper's 32.67. The reward for very small token budgets is louder than the raw-score gap.\n\n## More tokens can hurt\n\nThe ablation table shows compression is not a case of more being better. As injected tokens rise from 1 to 64, overall scores run 31.0 (1 token), 31.6 (2), 31.4 (4), 32.8 (8), 35.8 (16), 30.0 (32), and 34.2 (64). Thirty-two tokens scores below eight, and sixty-four still trails sixteen. The paper's explanation is the query-conditioned intrinsic rank r: the compression budget only needs to match r, and on LongBench-v2 r ≤ 16, so exceeding it adds no information and only makes optimization harder.\n\n## Not one team's idea\n\nAt least three groups are now working the pixel-for-text route. DeepSeek-OCR reported in October 2025 that a page of text compresses to roughly 100 visual tokens, keeping 97% OCR precision below a 10x compression ratio and about 60% at 20x, and on OmniDocBench surpassing GOT-OCR2.0 (256 tokens per page) using only 100 vision tokens. PixelRAG, from June 2026, retrieves web pages as screenshots directly and does better on text-centric benchmarks like NQ and SimpleQA, improving accuracy by up to 18.1% over text baselines and cutting token cost by up to 3x at lower resolution. RAGOCR, in August, trains a GRPO-based compressor to assign each passage a dynamic resolution, beating naive RAG by over 15% in accuracy with one-eighth the input tokens; its learned compression ratios double as reranking scores that outperform Qwen3-Reranker-4B.\n\nOpticalRAG belongs to that line but pushes the compression far harder, to 16 tokens.\n\n## Three limits before you ship this\n\nThe paper is explicit: evaluation covers **multiple-choice questions only**, with open-ended generation unverified; a fixed 16-token prefix budget may not suffice for questions requiring evidence spread across distant document segments; and adaptive prefix budgets are left to future work. The repository provides inference code and checkpoints but no training workflow, so reproducing training means building it yourself.\n\nThis is closer to a proof of direction than something to copy into production. But the question it raises is worth taking seriously for anyone doing context compression: we keep asking how to make text smaller, when maybe the better question is whether the information can be carried in a different modality.\n","opticalrag-pixel-space-rag-compression","2026-11-13T03:20:00Z","2026-11-13T03:18:46.230025Z","2026-11-13T03:18:46.230045Z",true,"agent",168,[39,49,58],{"slug":40,"tag_slug":41,"title_zh":42,"title_en":43,"intro_zh":44,"intro_en":45,"id":46,"is_active":35,"created_at":47,"modified_at":48},"image-generation","text-to-image","图像生成：从文生图到逐框编辑的模型竞赛","Image Generation: from text-to-image to frame-level editing","文生图已是红海，战场正转向编辑与可控性：FLUX.2\u002F3 用开源和量化把生图搬进消费级显卡，Qwen-Image、Seedream 等国产模型紧咬不放，GPT Image 2 则把推理能力带进像素。本专题追踪图像生成与编辑模型的发布、架构演进、加速量化与商业化数据，看「好图」如何变成「可用的图」。","Text-to-image is a crowded race, and the frontier has moved to editing and control: FLUX.2\u002F3 push open weights and quantization onto consumer GPUs, Qwen-Image and Seedream close in from China, and GPT Image 2 brings reasoning into pixels. This topic tracks model releases, architecture shifts, acceleration work, and the business numbers behind \"pretty\" becoming \"usable\".","52de7c59-7864-4569-8ae7-3d656d26a1a1","2026-11-20T07:20:23.534506Z","2026-11-20T07:20:23.534527Z",{"slug":50,"tag_slug":50,"title_zh":51,"title_en":52,"intro_zh":53,"intro_en":54,"id":55,"is_active":35,"created_at":56,"modified_at":57},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":59,"tag_slug":59,"title_zh":60,"title_en":61,"intro_zh":62,"intro_en":63,"id":64,"is_active":35,"created_at":65,"modified_at":66},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":68},[69,74,79,84,89,94],{"id":70,"title":71,"news_slug":72,"published_at":73},"0a4281e9-69b1-43f0-9e7b-78f0cd7ea4c2","医疗小模型回答砍72.8%,准确率反升3.2点","peca-conciseness-qlora-medical","2026-11-15T15:30:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"7984acc5-d045-4258-8da7-daf8ff25995b","一字节一token:27B模型词表砍到384的账本","qwen3-8-27b-byteify-vocab-384","2026-11-12T21:10:40+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"8d830f21-a44f-45b8-be6c-eed523d060dc","投机解码的天花板是分词器:3.4M参数换2.4倍","speculative-decoding-bpe-tokenizer-ceiling","2026-11-11T01:30:00+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"4f151ed3-74d5-4dc7-99d0-ac03df4c1b08","自一致别整条重采:定位错步,token至多省41%","stepback-error-localized-resampling","2026-11-08T13:20:00+00:00",{"id":90,"title":91,"news_slug":92,"published_at":93},"77a567e5-2209-4981-882b-e2a3077a0012","先猜再听:全双工语音模型把投机解码搬到音频到来之前","speculative-listening-full-duplex-speech-decoding","2026-11-06T09:05:00+00:00",{"id":95,"title":96,"news_slug":97,"published_at":98},"2983ae2f-a24f-40ae-a348-6f2c74443b19","CoSTA压缩语音token:口语对话生成提速1.7倍","costa-compressed-speech-token-adaptation","2026-10-31T15:13:09+00:00"]