[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-baidu-unlimited-ocr-rswa-constant-kv":3,"news-related-66079e92-3544-45b0-abeb-31d628220449":36},{"id":4,"title":5,"summary":6,"content":6,"original_url":7,"source_id":8,"tags":9,"translations":23,"news_slug":29,"published_at":30,"created_at":31,"modified_at":32,"is_published":33,"publish_type":34,"image_url":13,"view_count":35},"66079e92-3544-45b0-abeb-31d628220449","百度 Unlimited OCR：把端到端文档解析推进「一次性长文档」时代，R-SWA 把 KV 缓存压成常数","\n**摘要**：端到端 OCR 一直被「KV 缓存随输出线性增长」卡在十页以内——每多生成一个字，显存和延迟就多一份。百度团队把视角拉回人抄书时的「工作记忆」，提出 Reference Sliding Window Attention（R-SWA），在 DeepSeek OCR 的基座上把 KV 缓存压成常数。一夜之间，模型能一次前向解析几十页 PDF，还顺手在 OmniDocBench v1.5 上把 DeepSeek OCR 又甩开 6 个百分点。\n\n---\n\n6 月 22 日，百度在 GitHub、arXiv（2606.23050）、Hugging Face、ModelScope 同步开源 Unlimited OCR Works：3B 总参、500M 激活的 MoE 端到端模型，MIT 协议，发布次日登顶 GitHub Trending 与 Hugging Face 多模态趋势榜。底层仍是 DeepSeek OCR 的 DeepEncoder——16× token 压缩把一张 1024×1024 的 PDF 页压成 256 个视觉 token，多页推理的前缀负担就此稳住。\n\n真正的手术在解码端。R-SWA 给「参考 token（视觉 + prompt）」和「最近 n 个输出 token（默认 128）」开了两条独立通道：\n\n- **参考段** L_m 长度只跟图像分辨率相关，编码一次后全解码过程静态不变，视觉特征不会被自己的输出「磨糊」；\n- **解码段**以固定容量队列形式滑出，老 token 的 KV 不断被弹出，**总缓存大小收敛到 L_m + n 这个常数**。\n\n论文给出的对照非常直观：标准 MHA 的 cache 是 L_m + T（线性增长），R-SWA 是 L_m + min(n,T) ≤ L_m + n（常数），当 T 远大于 n 时，cache ratio 趋近 0。Flash Attention v3 的实测 kernel 延迟也变成了一条平直的线，而 DeepSeek OCR 每跨过 KV 对齐边界就会出现一次尖峰。\n\n在 OmniDocBench v1.5 上，Unlimited OCR 拿到 93%，比 DeepSeek OCR 强 6 个百分点；公开评测 v1.6 综合分达 93.92%，刷新端到端 OCR 公开纪录。最大单次推理长度 32K，几十页 PDF 一次 forward pass 即可吐完。发布 6 天内 vLLM 官方补上 recipe（vllm\u002Fvllm-openai:unlimited-ocr 镜像）、SGLang 同步放出 streaming server、AK 在 Hugging Face Spaces 搭了 demo。\n\n更大的信号藏在论文结尾：R-SWA 是一种「参考型解析注意力」，对参考 token 静态、对输出 token 滑动——这种结构天然契合 ASR（参考 token 是音频段）、长文档翻译、代码库级别的多文件重构等所有「长输出 + 固定参考」任务。它不只是一次 OCR 升级，而是一类新的 attention 范式：从硬扛超长 context window，转向把 attention 本身变得对长输出友好。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2606.23050","7437aeb9-930c-4866-a2e9-48003c1a792b",[10,14,17,20],{"id":11,"name":12,"slug":12,"description":13,"color":13},"01598627-1ea6-4b27-a5d8-874971571a71","llm",null,{"id":15,"name":16,"slug":16,"description":13,"color":13},"7e89b5cc-57db-4f37-bc6d-28919a73931c","model-release",{"id":18,"name":19,"slug":19,"description":13,"color":13},"499f4b56-819d-49a3-9609-33e775143b86","multimodal",{"id":21,"name":22,"slug":22,"description":13,"color":13},"b9bd9039-fcdb-41a8-b85b-fc1587def2b9","open-source",[24],{"id":25,"lang":26,"title":27,"summary":28,"content":13},"f18ab114-6acb-46e0-95aa-4d61c33c66b7","en","Baidu Unlimited OCR handles whole long documents in one pass","End-to-end OCR has long been stuck within ten pages because the KV cache grows linearly with output length — every additional generated character adds another slice of VRAM and latency. The Baidu team took a step back to how humans copy a book, with \"working memory,\" and proposed Reference Sliding Window Attention (R-SWA), which compresses the KV cache to a constant on top of DeepSeek OCR's base. Overnight, the model can parse dozens of PDF pages in a single forward pass, and on OmniDocBench v1.5 it pulls another 6 points ahead of DeepSeek OCR.\n\nOn June 22, Baidu open-sourced Unlimited OCR Works simultaneously on GitHub, arXiv (2606.23050), Hugging Face, and ModelScope — a 3B-total \u002F 500M-active MoE end-to-end model under MIT, hitting GitHub Trending and the Hugging Face multimodal-trend chart the day after release. The base is still DeepSeek OCR's DeepEncoder — 16× token compression squashes a 1024×1024 PDF page into 256 visual tokens, locking down the prefix cost of multi-page inference.\n\nThe real surgery is on the decoding side. R-SWA opens two independent channels for \"reference tokens (visual + prompt)\" and \"the most recent n output tokens (default 128)\": the reference segment L_m is length-bound by image resolution only, and once encoded it stays static throughout decoding so visual features are never \"smeared\" by their own output; the decoding segment is a fixed-capacity queue that slides out, so old tokens' KV keeps getting evicted, and the total cache size converges to the constant L_m + n.\n\nThe paper's contrast is intuitive: a standard MHA cache is L_m + T (linear growth); R-SWA is L_m + min(n, T) ≤ L_m + n (constant). When T is much larger than n, the cache ratio approaches 0. The Flash Attention v3 kernel latency also flattens into a horizontal line, while DeepSeek OCR spikes every time it crosses a KV-alignment boundary.\n\nOn OmniDocBench v1.5 Unlimited OCR scores 93%, 6 points ahead of DeepSeek OCR; on public eval v1.6 it hits a comprehensive 93.92%, refreshing the public record for end-to-end OCR. The longest single inference is 32K — dozens of PDF pages parsed in one forward pass. Within six days of release, vLLM added a recipe (vllm\u002Fvllm-openai:unlimited-ocr image), SGLang shipped a streaming server in parallel, and AK put up a demo on Hugging Face Spaces.\n\nA larger signal hides in the paper's close: R-SWA is a \"reference-style parsing attention\" that is static for reference tokens and sliding for output tokens. That structure is a natural fit for ASR (where reference tokens are audio segments), long-document translation, and codebase-scale multi-file refactoring — every \"long output + fixed reference\" task. It's not just an OCR upgrade; it's a new attention paradigm: instead of bracing for ever-longer context windows, the attention itself becomes friendly to long output.","baidu-unlimited-ocr-rswa-constant-kv","2026-06-29T08:00:00Z","2026-06-29T08:08:30.866937Z","2026-08-19T02:08:40.142862Z",true,"agent",97,{"items":37},[38,43,48,53,58,63],{"id":39,"title":40,"news_slug":41,"published_at":42},"40095b51-97b0-4fd4-9b1d-f636c970572e","阿里 Qwen 团队发布 Qwen3.8-Max:2.4 万亿参数 MoE 模型首度开放权重","qwen3-8-max-2-4t-moe-open-weights","2026-08-07T02:00:00+00:00",{"id":44,"title":45,"news_slug":46,"published_at":47},"804b44fb-66c6-4355-83a4-b3a03a776d2a","Inkling-Small 开放权重：12B 激活参数换来更高 Agent 效率，也暴露事实性短板","inkling-small-multimodal-moe-efficiency","2026-08-05T16:32:13+00:00",{"id":49,"title":50,"news_slug":51,"published_at":52},"a151db0c-d832-4df2-ac03-2d4e58b26e99","Kimi K3 跑通 MiniTriton:Moonshot 让 LLM 第一次从零编译出自己的 GPU 编译器","kimi-k3-minitriton-gpu-compiler","2026-07-26T14:00:00+00:00",{"id":54,"title":55,"news_slug":56,"published_at":57},"dfc3dec4-2211-4c7e-b6ff-9e0d9a479ec4","微软与 Mistral 签下数十亿美元协议:Vera Rubin GPU 上的「欧洲主权云」开始落地","microsoft-mistral-vera-rubin-sovereign","2026-07-22T02:00:00+00:00",{"id":59,"title":60,"news_slug":61,"published_at":62},"cf01282f-8a64-49a8-a608-9b806ccfbea3","Mira Murati 实验室 Inkling 开源：975B MoE 不卷\"最强\"，押注\"可定制\"","thinking-machines-inkling","2026-07-15T22:00:00+00:00",{"id":64,"title":65,"news_slug":66,"published_at":67},"1a27bedc-012d-4d63-85e1-ddc57aabd8bf","ByteDance UniVR 让模型「在视觉空间里思考」：34B 参数逼近 Gemini 3 Pro + Nano Banana 2","bytedance-univr-34b","2026-07-14T12:10:00+00:00"]