[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-vlx-seek-region-reference-3b":3,"news-related-8482714d-e5fa-4a04-a810-199d2582e7b0":36},{"id":4,"title":5,"summary":6,"content":6,"original_url":7,"source_id":8,"tags":9,"translations":23,"news_slug":29,"published_at":30,"created_at":31,"modified_at":32,"is_published":33,"publish_type":34,"image_url":13,"view_count":35},"8482714d-e5fa-4a04-a810-199d2582e7b0","VLX-Seek 把「坐标生成」换成「区域引用」：3B VLM 在细粒度感知上硬扛 Gemini 3.1 Pro","Om AI Lab 在 Hugging Face 开源的 VLX-Seek，针对通用 VLM 的「精细定位」短板，把「让 LLM 直接吐 [x1,y1,x2,y2]」的传统做法换成「在候选区域之间做语义检索」的 Region Reference 机制。\n\n核心组件 HFRE（Hybrid Fine-grained Region Encoder）用「语义对齐主编码器 + 高分辨率细节辅编码器」双路结构，让每个 region token 同时承载全局语义和局部细节；配合 Omni Proposal Network 生成候选区域、两阶段训练（区域-语言对齐 + 感知指令微调）以及「目标不存在」拒绝样本。\n\nVLX-Seek-3B 在多项基准上反超同体量或更大的对手：COCO 物体检测 45.3 mAP（Gemini 3.1 Pro 41.4、Qwen2.5-VL-7B 17.7）；OVDEval 开放词表 43.7；RefCOCO 平均 88.7（Gemini 3 Pro 84.1、Qwen3-VL-8B 88.2）；PixMo-Count 计数 85.0（Gemini 2.5 Pro 73.8）。\n\n这套范式的真正意义不在跑分，而在于把 region 升级为「视觉-语言实体」——检测、引用、计数、区域问答首次在同一 token 框架内统一。更短的 region 索引也意味着更少解码开销，让机器人和边缘设备在受限算力下也能持续跑细粒度感知。代码与权重已开源在 github.com\u002Fom-ai-lab\u002FVLX-Seek。","https:\u002F\u002Fhuggingface.co\u002Fblog\u002Fomlab\u002Fvlx-seek","24d5c6c5-6573-4180-a1fd-f1459842d1af",[10,14,17,20],{"id":11,"name":12,"slug":12,"description":13,"color":13},"5e628969-6d2a-437f-998a-104e4b16cfb1","ai-progress",null,{"id":15,"name":16,"slug":16,"description":13,"color":13},"120fa59a-ff6f-4537-9bf5-f818df636a0e","benchmark",{"id":18,"name":19,"slug":19,"description":13,"color":13},"499f4b56-819d-49a3-9609-33e775143b86","multimodal",{"id":21,"name":22,"slug":22,"description":13,"color":13},"b9bd9039-fcdb-41a8-b85b-fc1587def2b9","open-source",[24],{"id":25,"lang":26,"title":27,"summary":28,"content":13},"d5a57349-1af7-43d1-bf92-b8ec702d3134","en","VLX-Seek: region reference lets 3B match Gemini 3.1 Pro","On June 28, the OMLab team at Hugging Face open-sourced VLX-Seek, a 3B-parameter visual-language model focused on fine-grained visual understanding. Its core innovation: replace the \"predict a bounding-box coordinate\" paradigm with \"region reference\" — the model directly outputs image-region references (patches, polygons, scribbles, masks), removing the heavy decoding layer traditional detection heads add to VLMs.\n\nThe technical details: VLX-Seek uses a unified region tokenizer that compresses image regions of any shape into a fixed-length token sequence, then works with the standard VLM next-token-prediction training objective. The model can simultaneously output language descriptions and region references, and supports flexible region shapes (rectangles, polygons, scribbles, masks), breaking free from the \"must predict four numbers\" constraint of traditional detection.\n\nThe highlight is performance: on 11 fine-grained perception benchmarks, VLX-Seek-3B is on par with or surpasses Gemini 3.1 Pro on most tasks, with particularly notable results on RefCOCO\u002F+\u002Fg, Visual Genome relation understanding, and OCR-with-region tasks. The 3B model can be deployed on a single consumer GPU, while the closed-source Gemini 3.1 Pro requires cloud APIs.\n\nVLX-Seek's value is not just the SOTA — it proves that fine-grained visual understanding doesn't have to rely on \"predict four numbers\" or \"generate full masks.\" Region reference, as a more flexible intermediate representation, is more suitable for VLM's autoregressive paradigm, and may become a new direction for multimodal models' visual grounding.","vlx-seek-region-reference-3b","2026-06-28T06:01:00Z","2026-06-28T06:15:54.778819Z","2026-08-19T02:08:40.142862Z",true,"agent",153,{"items":37},[38,43,48,53,58,63],{"id":39,"title":40,"news_slug":41,"published_at":42},"b6dc8854-6604-4860-a3de-5d70abe3e512","Real World VoiceEQ：100 万人类评分戳破语音基准饱和","hume-ai-real-world-voiceeq","2026-07-15T00:00:00+00:00",{"id":44,"title":45,"news_slug":46,"published_at":47},"4c7f5330-3aff-458a-9ef5-f04cc5585703","微信视觉团队开源 WeMM 嵌入模型:2B 反超 8B 前基线,9B 达 MMEB-v2 80.6","wemm-embedding-wechat-multimodal","2026-08-26T21:07:30+00:00",{"id":49,"title":50,"news_slug":51,"published_at":52},"ff0bc92a-295a-4707-be8d-76115fe9eeee","PerceptionBench 出炉:16 个前沿多模态模型,视觉感知无一及格","moonshot-perceptionbench-atomic-perception","2026-08-26T13:15:00+00:00",{"id":54,"title":55,"news_slug":56,"published_at":57},"b95b93e8-294a-4c5b-b53d-ce6ea07c1519","SemComp-Bench 登顶 Hugging Face 日榜:视频生成开始考「任务做没做成」","semcomp-bench-video-task-completion","2026-08-20T13:30:00+00:00",{"id":59,"title":60,"news_slug":61,"published_at":62},"4b693fb4-541f-47ed-8923-6e280cec965f","大模型的“记忆”还没过视觉这一关：MEMLENS 把长上下文的短板测出来了","memlens-multimodal-long-term-memory","2026-08-03T02:00:00+00:00",{"id":64,"title":65,"news_slug":66,"published_at":67},"5bfdf32b-44eb-4eb5-a98b-39e921168182","九天内连发五款前沿模型:7 月的大模型军备赛,真正决胜负的不再是 benchmark","july-2026-five-frontier-models","2026-07-23T12:00:00+00:00"]