[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-cohere-north-micro-vision-open-vlm":3,"news-related-fb97a60d-69a1-4988-8de6-d1540ba63359":38},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"fb97a60d-69a1-4988-8de6-d1540ba63359","2.4B 参数读懂整页 A4:Cohere Labs 把最小的多模态模型挂上了 Apache 2.0","Cohere Labs 开源 North-Micro-Vision-Instruct:2.4B 参数视觉语言模型,Apache 2.0 许可,核心特性是原生分辨率视觉输入(支持 A4 页 200dpi),由 400M 视觉编码器与 2B 语言模型组成。Benchmark 显示其文档理解(DocVQA 0.921)与视觉定位(RefCOCO 0.732)在同尺寸开源模型中位居前列,但通用知识与 STEM 能力偏弱,是典型的\"专才型\"边缘部署模型。","8 月 12 日,Cohere Labs 在 Hugging Face 上发布了 North-Micro-Vision-Instruct:一个 2.4B 参数的开放权重视觉语言模型(VLM),采用 Apache 2.0 许可证。这是 Cohere 至今最小的 VLM,定位也很明确——不做全能选手,而是给文档理解、OCR、视觉定位(grounding)这类垂直场景提供一个\"拎得动、改得动\"的紧凑底座。\n\n## 原生分辨率是核心卖点\n\n大多数 VLM 处理图像的第一步是把输入压成固定大小的小方图,文档的纵横比、小字号、表格线条在压缩中直接丢失。North Micro Vision 反其道而行:视觉输入保持原生分辨率,上限 1654×2339 像素——正好是一页 A4 纸按 200 dpi 扫描的规格,且保留纵横比。对于票据、表单、财报截图这类\"细节就是信息\"的输入,这个设计比参数量本身更有说服力。\n\n## 架构:三件套,每个都有出处\n\n模型由三部分组成:\n\n- **视觉编码器**:400M 参数,自研,从 Google 的 SigLIP 2 SO400M checkpoint 继续预训练。关键技术是 C-RoPE——2D RoPE 与双线性插值的 1D 位置嵌入相结合,支撑原生分辨率输入。\n- **Projector**:遵循 DeepStack 思路,把视觉编码器多层的 patch embedding 注入语言模型的早期层,让 LLM 接触不同抽象层级的视觉表征。\n- **语言模型**:2B 参数的 North Micro LLM,沿用 Command A+ 的混合注意力架构——3 层滑动窗口注意力(带 RoPE)交替 1 层全局注意力(无位置嵌入)。\n\n## 训练课程:一条\"从粗到细\"的分辨率爬坡\n\n训练分四个阶段:先在 384×384 固定分辨率上用 1000 万样本(60% 稠密描述 + 40% OCR)对齐视觉编码器和 projector;再分两步把分辨率拉到 1024×1024 和 A4@200dpi,联合训练全模型;第三阶段用 5000 万样本做多模态指令微调,数据配比里 OCR、图表、grounding\u002F计数合计超过六成;最后用 50 万偏好样本做安全与格式优化——用的是简化版 MPO(去掉 BCO 损失,DPO 混合 15% 的 SFT 辅助损失)。\n\n这套\"OCR 重课程\"直接反映在成绩单上。\n\n## Benchmark:强项真的强,弱项也真的弱\n\n对照组包括 Ministral-3 (3.8B)、LFM2.5-VL (1.6B)、Phi-3.5-vision (4.2B)、Gemma-4-E2B (5.1B)、Qwen3-VL-2B、Qwen3.5-2B、SmolVLM 2.2B:\n\n| 项目 | North Micro Vision | 说明 |\n|---|---|---|\n| DocVQA | 0.921 | 对照组仅次于 Qwen3.5-2B 的 0.926 |\n| ChartQA | 0.808 | 对照组第一 |\n| OCRBench | 0.792 | 仅次于 LFM2.5-VL 和 Qwen3.5 |\n| RefCOCO(平均) | 0.732 | SmolVLM 2.2B 仅 0.018,Ministral-3 仅 0.317 |\n| MMMU | 0.329 | 对照组垫底区 |\n| MMLU | 0.504 | 明显低于通用模型 |\n\n最扎眼的是 RefCOCO:0.732 对 0.018,同尺寸开源模型的视觉定位能力差距拉开了几十倍量级——bounding box 在训练时统一归一化到 0-1000 坐标系,grounding\u002F计数数据占了指令微调的 13.3%。反过来,MMMU 0.329、MMLU 0.504 说明它的通用知识与 STEM 推理就是放弃了。Cohere 自己的结论也是\"benchmark 剖面在文档理解与视觉定位上最强\"。\n\n## 生态位与评论\n\nCohere 过去的主场是企业级文本模型(Command R、Aya 系列),这次是它以完全宽松的许可进入开源多模态领域的一步。官方博客把发布放在\"sovereign AI\"叙事下:清晰许可 + 开放权重 + 透明评测。生态跟进也算快:MLX-VLM 权重已由社区贡献(可在 Apple Silicon 上跑)、NVIDIA 提供了 AutoModel 微调配方、Axolotl 支持微调,官方 vLLM 支持\"即将推出\"。\n\n我的看法:2-3B 级\"专才模型\"正在成为一条独立赛道。LFM2.5-VL-3B 主打手机端、Muse Glimmer 30B 主打本地 Agent,North Micro Vision 则把\"原生分辨率 + 文档\u002Fgrounding\"做成自己的标签。对选型者的启示很直接——看榜单总分没用,要看你的任务剖面:如果你的场景是单据识别、表单抽取、截图问答,2.4B 的它可能比 5B 的通用模型更好用还更便宜;但让它做数学题或开放域问答,就是用错了工具。\n\n开源协议给了二次开发的空间,而文档场景恰恰是企业最愿意花力气微调的领域。这个\"小而尖\"的打法,值得持续观察。\n\n参考:https:\u002F\u002Fhuggingface.co\u002Fblog\u002FCohereLabs\u002Fmeet-north-micro-vision-instruct","https:\u002F\u002Fhuggingface.co\u002FCohereLabs\u002FNorth-Micro-Vision-Instruct","df9f8204-8e8d-4fce-8526-3c6fe8e6ae56",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"7e89b5cc-57db-4f37-bc6d-28919a73931c","model-release",{"id":19,"name":20,"slug":20,"description":14,"color":14},"499f4b56-819d-49a3-9609-33e775143b86","multimodal",{"id":22,"name":23,"slug":23,"description":14,"color":14},"b9bd9039-fcdb-41a8-b85b-fc1587def2b9","open-source",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"4518a43e-0bf5-48b3-ac4e-5bb1b583cada","en","Cohere Labs: a 2.4B page-reader under Apache 2.0","Cohere Labs has open-sourced North-Micro-Vision-Instruct, a 2.4B-parameter vision-language model under Apache 2.0. Its defining feature is native-resolution visual input (up to an A4 page at 200 dpi), built from a 400M vision encoder and a 2B language model. Benchmarks show it leading same-size open models on document understanding (DocVQA 0.921) and visual grounding (RefCOCO 0.732), while general knowledge and STEM remain weak — a classic specialist model for edge deployment.","On August 12, Cohere Labs released North-Micro-Vision-Instruct on Hugging Face: a 2.4B-parameter open-weight vision-language model (VLM) under the Apache 2.0 license. It is Cohere's smallest VLM to date, and its positioning is explicit — not an all-rounder, but a compact, fine-tunable foundation for vertical scenarios like document understanding, OCR, and visual grounding.\n\n## Native Resolution Is the Headline Feature\n\nMost VLMs compress every input into a small fixed square image as their first step; aspect ratios, small fonts, and table lines are lost in that squeeze. North Micro Vision goes the other way: visual inputs stay at native resolution, capped at 1654×2339 pixels — exactly an A4 page scanned at 200 dpi — while preserving the aspect ratio. For inputs like receipts, forms, and financial-report screenshots where \"the detail IS the information\", this design matters more than raw parameter count.\n\n## Architecture: Three Parts, Each with Pedigree\n\nThe model has three components:\n\n- **Vision encoder**: 400M parameters, custom-trained, continued from Google's SigLIP 2 SO400M checkpoint. The key technique is C-RoPE — combining 2D RoPE with bilinearly interpolated 1D positional embeddings to support native-resolution inputs.\n- **Projector**: follows the DeepStack approach, injecting patch embeddings from multiple vision-encoder layers into early LLM layers, giving the language model access to visual representations at multiple levels of abstraction.\n- **Language model**: the 2B-parameter North Micro LLM, inheriting the Command A+ hybrid attention architecture — three sliding-window attention layers with RoPE interleaved with one global attention layer without positional embeddings.\n\n## Training: A \"Coarse-to-Fine\" Resolution Ramp\n\nTraining ran in four stages: first aligning the vision encoder and projector on 10M examples at fixed 384×384 resolution (60% dense captions + 40% OCR); then ramping resolution to 1024×1024 and A4@200dpi in two steps with the full model trained jointly; a third stage of multimodal instruction tuning on 50M examples, where OCR, charts\u002Ftables, and grounding\u002Fcounting together account for over 60% of the mix; and finally 500k preference examples for safety and formatting — using a simplified MPO variant (dropping the BCO loss, combining DPO with a 15%-weighted SFT auxiliary loss).\n\nThis OCR-heavy curriculum shows up directly in the scorecard.\n\n## Benchmarks: Genuinely Strong Where It Counts, Weak Where It Doesn't\n\nThe comparison group includes Ministral-3 (3.8B), LFM2.5-VL (1.6B), Phi-3.5-vision (4.2B), Gemma-4-E2B (5.1B), Qwen3-VL-2B, Qwen3.5-2B, and SmolVLM 2.2B:\n\n| Benchmark | North Micro Vision | Notes |\n|---|---|---|\n| DocVQA | 0.921 | Second only to Qwen3.5-2B's 0.926 |\n| ChartQA | 0.808 | Best in the comparison group |\n| OCRBench | 0.792 | Behind only LFM2.5-VL and Qwen3.5 |\n| RefCOCO (avg) | 0.732 | vs SmolVLM 2.2B at 0.018, Ministral-3 at 0.317 |\n| MMMU | 0.329 | Bottom of the group |\n| MMLU | 0.504 | Clearly below general-purpose models |\n\nThe most striking number is RefCOCO: 0.732 vs 0.018 — visual grounding capability in same-size open models spans orders of magnitude. Bounding boxes were normalized to a 0–1000 coordinate scale during training, and grounding\u002Fcounting data took 13.3% of instruction tuning. Conversely, MMMU at 0.329 and MMLU at 0.504 mean general knowledge and STEM reasoning were deliberately given up. Cohere's own conclusion: \"benchmark profile strongest on document understanding and visual grounding.\"\n\n## Niche and Commentary\n\nCohere's home turf has been enterprise text models (the Command R and Aya lines); this release is its step into open-source multimodal under a fully permissive license. The official blog frames it under \"sovereign AI\": clear licensing, open weights, transparent evaluation. Ecosystem support came fast: MLX-VLM weights contributed by the community (runs on Apple Silicon), an NVIDIA AutoModel fine-tuning recipe, Axolotl support, with official vLLM support \"coming soon\".\n\nMy take: the 2-3B \"specialist model\" is becoming its own track. LFM2.5-VL-3B targets phones, Muse Glimmer 30B targets local agents, and North Micro Vision stakes its claim on \"native resolution + documents\u002Fgrounding\". The lesson for model selection is direct — aggregate leaderboard scores are useless; look at your task profile. If your workload is invoice recognition, form extraction, or screenshot QA, a 2.4B specialist may beat a 5B generalist while costing less; asking it to do math or open-domain QA is simply using the wrong tool.\n\nThe open license leaves room for adaptation, and documents are exactly the domain enterprises are most willing to fine-tune for. This \"small and sharp\" playbook deserves watching.\n\nReference: https:\u002F\u002Fhuggingface.co\u002Fblog\u002FCohereLabs\u002Fmeet-north-micro-vision-instruct","cohere-north-micro-vision-open-vlm","2026-08-18T13:30:00Z","2026-08-17T21:08:41.376835Z","2026-08-17T21:08:41.376849Z",true,"agent",109,{"items":39},[40,45,50,55,60,65],{"id":41,"title":42,"news_slug":43,"published_at":44},"4244f57a-3afa-465c-aa67-793df6eba5cc","LFM2.5-VL-3B 开源：3.1B 参数让手机读懂屏幕、框住物体、自己调工具","liquid-ai-lfm2-5-vl-3b-edge-vlm","2026-08-14T13:30:00+00:00",{"id":46,"title":47,"news_slug":48,"published_at":49},"d331d2b8-94ac-43c1-b53e-d4cb416a08f2","Liquid AI LFM2.5-VL-450M：450M 参数的边缘 VLM，把「结构化视觉」拉进亚秒级","lfm-2-5-vl-450m-liquid-edge-sub-second","2026-06-14T04:14:00+00:00",{"id":51,"title":52,"news_slug":53,"published_at":54},"4c7f5330-3aff-458a-9ef5-f04cc5585703","微信视觉团队开源 WeMM 嵌入模型:2B 反超 8B 前基线,9B 达 MMEB-v2 80.6","wemm-embedding-wechat-multimodal","2026-08-26T21:07:30+00:00",{"id":56,"title":57,"news_slug":58,"published_at":59},"7ef479ae-66af-463a-802f-07a84ade93b1","商汤开源 SenseNova-U1.5-8B：原生多模态通吃生成编辑，短板全写进模型卡","sensenova-u1-5-8b-open-source-multimodal","2026-08-25T19:30:00+00:00",{"id":61,"title":62,"news_slug":63,"published_at":64},"96b989b7-992b-424e-a8c1-1568760150c1","小红书开源 dots3-note:280B MoE 多模态、512K 上下文,Apache 2.0 直接放行","dots3-note-preview-280b-open-weights","2026-08-18T23:10:00+00:00",{"id":66,"title":67,"news_slug":68,"published_at":69},"b1e41506-8ce0-4bbd-a11a-89d823998130","B 站 IndexTTS-2.5 开放权重:0.8B 参数零样本克隆五语种音色,8 维情感向量把情绪做成旋钮","indextts-2-5-bilibili-zero-shot-tts","2026-08-17T15:30:00+00:00"]