[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-image-prompt-packaging-multimodal-35-91pct":3,"news-related-c84f5d0b-65d1-411c-8f76-75c301a748b2":36},{"id":4,"title":5,"summary":6,"content":6,"original_url":7,"source_id":8,"tags":9,"translations":23,"news_slug":29,"published_at":30,"created_at":31,"modified_at":32,"is_published":33,"publish_type":34,"image_url":13,"view_count":35},"c84f5d0b-65d1-411c-8f76-75c301a748b2","多模态AI的token成本困局：Image Prompt Packaging带来推理降本新思路","商业大模型的按token计费模式，让多模态推理的成本问题变得日益尖锐。当一段财务分析需要附带七张图表、一份合同扫描附带密集的条款文字时，文本token的开销往往远超图片本身。如何在不修改模型的前提下降低多模态推理成本？arXiv上最新发表的研究提出了一个颇为巧妙的思路——Image Prompt Packaging（IPPg），把文字直接写进图片里，用视觉token替代文本token。\n\nIPPg的原理并不复杂：在图片的留白区域直接嵌入结构化文本，让大模型通过视觉通道读取文字信息，而非传统的文本token通道。由于主流商业模型对图片token和文本token的计价相同（均按输入token收费），一张图片在视觉编码后的token占用，往往远少于等效文本的token数。研究团队的测试覆盖了五个数据集、三款前沿模型（GPT-4.1、GPT-4o、Claude 3.5 Sonnet），横跨VQA和代码生成两类任务。\n\n结果显示，IPPg在最优配置下可实现35.8%至91.0%的推理成本削减。不过，效果高度依赖任务类型和模型：在CoSQL数据库理解任务上，GPT-4.1达到了准确率和成本的双赢；而Claude 3.5在部分VQA基准上反而出现了成本上升。失败模式分析揭示了明确的边界：空间推理、非英文输入、字符级操作（如精确计数）是IPPg最脆弱的场景；相对地，schema结构化任务（如数据库Schema理解）受益最大。\n\nIPPg的意义不在于取代现有的多模态方案，而在于揭示了一个长期被忽视的事实：多模态推理的成本优化，不止属于模型压缩和量化那条路径，prompt层面的路由同样有巨大空间。把文字送进视觉通道，本质上是对信息编码形式的一次重新审视——相同语义，不同载体，成本却可能天差地别。当然，这项工作基于商业API的定价结构，随着多模态模型定价体系的调整其实用性可能变化。此外，把文字写进图片这一操作本身也引入了新的工程复杂度——自动排版、字体选择、渲染质量控制，都是落地时必须考虑的成本。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2604.02492","7437aeb9-930c-4866-a2e9-48003c1a792b",[10,14,17,20],{"id":11,"name":12,"slug":12,"description":13,"color":13},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",null,{"id":15,"name":16,"slug":16,"description":13,"color":13},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":18,"name":19,"slug":19,"description":13,"color":13},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":21,"name":22,"slug":22,"description":13,"color":13},"499f4b56-819d-49a3-9609-33e775143b86","multimodal",[24],{"id":25,"lang":26,"title":27,"summary":28,"content":13},"fe75ace2-b0b4-4210-ac60-7c3cfe60e636","en","Image prompt packaging: a new way to cut multimodal token costs","Commercial LLMs' per-token pricing model has made multimodal inference cost an increasingly acute pain point. When a financial analysis comes with seven charts, or a scanned contract is packed with dense clause text, the text-token overhead often dwarfs the image itself. How do you cut multimodal inference cost without touching the model? A new study on arXiv proposes a clever angle — Image Prompt Packaging (IPPg) — embed text directly into the image, and use vision tokens to replace text tokens.\n\nThe principle is straightforward: in the empty space of an image, structured text is embedded, so the model reads it through the visual channel instead of the conventional text-token channel. Because mainstream commercial models charge the same for image tokens and text tokens (both billed as input tokens), a single image, once visually encoded, typically consumes far fewer tokens than its text equivalent. The team tested across five datasets and three frontier models (GPT-4.1, GPT-4o, Claude 3.5 Sonnet), spanning VQA and code-generation tasks.\n\nResults show that under optimal settings, IPPg achieves 35.8%-91.0% inference cost reduction. However, the effect is highly task- and model-dependent: on CoSQL database understanding, GPT-4.1 hits a sweet spot of accuracy and cost; on certain VQA benchmarks, Claude 3.5 actually sees cost increases. Failure-mode analysis reveals clear boundaries — spatial reasoning, non-English input, and character-level operations (e.g., precise counting) are IPPg's most fragile zones; structured-schema tasks (e.g., database schema understanding) benefit the most.\n\nThe significance of IPPg is not in replacing existing multimodal solutions, but in spotlighting a long-overlooked fact: multimodal inference cost optimization is not the sole province of model compression and quantization — prompt-level routing has just as much room. Routing text through the visual channel is essentially a fresh look at information encoding — same semantics, different carrier, dramatically different cost. That said, this work is built on commercial API pricing structures; its utility may shift as multimodal pricing evolves. And embedding text into images brings its own engineering complexity — auto-layout, font selection, and render-quality control are all real costs to weigh at deployment.","image-prompt-packaging-multimodal-35-91pct","2026-05-26T04:08:00Z","2026-05-26T04:12:05.754956Z","2026-08-19T02:08:40.142862Z",true,"agent",115,{"items":37},[38,43,48,53,58,63],{"id":39,"title":40,"news_slug":41,"published_at":42},"5e4ce9dc-9454-4e4c-997d-467617d00fee","打破高质量嵌入的「不可能三角」：ML-Embed 三维 Matryoshka 框架直击低资源语言痛点","ml-embed-3d-matryoshka-low-resource","2026-05-17T04:10:00+00:00",{"id":44,"title":45,"news_slug":46,"published_at":47},"ea444bd9-4683-486b-b606-c222d98f1ba7","标注即 rollout:南开 OraRL 把视频多模态 RL 训练成本砍半,9B 空间智能超 GPT-5","orarl-annotations-as-rollouts-video-rl","2026-08-26T17:10:00+00:00",{"id":49,"title":50,"news_slug":51,"published_at":52},"cac485ab-a429-4ebc-88c6-a1f924f978ff","AWS 开源 KeysAndValues:微调时就让模型学会“遗忘”,单张 A100 撑住 128K","aws-keysvalues-sparse-attention-finetuning","2026-08-26T05:20:00+00:00",{"id":54,"title":55,"news_slug":56,"published_at":57},"c94bdf86-5de9-49fe-8c98-0f5c47611bfe","SGLang v0.5.18 发布:大模型冷启动提速 2.38 倍,710 个 PR 都改了什么","sglang-v0-5-18-cold-start-2-38x","2026-08-24T23:15:00+00:00",{"id":59,"title":60,"news_slug":61,"published_at":62},"ce70384a-990b-4994-bfb6-27775be45661","TensorRT Edge-LLM 0.10.0：边端第一个统一的 C++ 多模态推理栈","tensorrt-edge-llm-0-10-multimodal-runtime","2026-08-23T00:00:00+00:00",{"id":64,"title":65,"news_slug":66,"published_at":67},"92433e6b-113a-4ada-af77-fbb8995a9850","LFM2.5-DSpark 开源:300M 草稿模型让端侧推理快 2.87 倍,输出零损耗","lfm2-5-dspark-draft-models","2026-08-21T21:10:00+00:00"]