[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-artificial-analysis-pipette-mobile-llm-benchmark":3,"topics-all":38,"news-related-89de53a6-5cee-40af-8190-1c22d628b738":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"89de53a6-5cee-40af-8190-1c22d628b738","23 个端侧 LLM 同台比:Artificial Analysis 把 iPhone 17 Pro 变成首个开放基准","Artificial Analysis 与 Liquid AI 推出首个手机端 LLM 基准,测 23 个量化后能跑进 8GB 内存的模型,LFM2.5-8B 端侧解码比 Qwen3.5-4B 快 2.4 倍。","## 背景:手机跑得动的 LLM,缺一份公开账\n\n过去两年,小尺寸 LLM(LFM2、Qwen3.5、Granite-4 等)都在讲一个故事:模型能在手机或笔记本上跑出可用质量。但\"跑得动\"和\"跑得稳\"之间的真实差距,从来没有公开账可看 —— 模型方放的是 fp16 跑分,设备方放的是营销视频,中间缺一道独立测试。Artificial Analysis 联合 Liquid AI 推出的 **Pipette** 基准,补的就是这一环:把\"模型 + 量化 + runtime + 设备 + workload\"五元组拆开实测,公开给所有人看([artificialanalysis.ai](https:\u002F\u002Fartificialanalysis.ai\u002Fhardware-inference-stack\u002Fmobile-phones))。\n\n这套基准的出现节点很关键。8 月底 Google 刚刚因为 AI 数据中心抢走 DRAM,要求 Android 应用开发商在 2027 年 2 月前压缩内存占用;同期微软内部数据显示,AI 工程师账单失控,28 天烧 2.8 万美元。两条新闻指向同一个事实:在数据中心 AI 越跑越贵的同一时间,业内对\"推理能不能从云挪到端\"的需求是真实的。但端侧推理不是装上 APK 就能跑 —— 量化精度、运行时、KV cache 内存预算、上下文长度,每一个都会决定同一个模型在你设备上是不是真能用。\n\n## 基准的玩法\n\n测试对象是量化后能装进 8GB 内存的模型(含 8K 上下文的 KV cache),采用 Q4_K_M 或更小的 GGUF 量化,推理框架是 llama.cpp。设备覆盖 iPhone 17 Pro(12GB)、Galaxy S26 Ultra(12GB)、MacBook Pro M5 Max,以及即将上线的 AMD Ryzen AI Max+ 395 + Radeon 8060S。评测维度包括 BFCL(工具调用子集,640 题)、IFBench(指令遵循,294 题)、AA-Omniscience(知识与反幻觉,6000 题)、GPQA Diamond(科学推理,198 题)、MATH-500(数学,500 题),五项等权平均给出综合智力分数;推理侧用 1024 token prompt + 256 token response,记录 end-to-end 时延、prefill\u002Fdecode 吞吐、峰值 RAM。每个评测跑 5 次取 pass@1,推理跑之前还要做平台特定的\"温度和负载稳定\"检查,降低环境噪声。\n\n首批发布覆盖 23 个模型,涵盖 Liquid AI 全系 LFM2.5、Qwen3.5\u002F3.6 系列、Gemma 4 E2B\u002FE4B\u002F12B、IBM Granite 4.0\u002F4.1、Mistral Ministral 3、Llama 3.2 1B\u002F3B、TII Falcon-H1R-7B、Allen AI Olmo-3-7B-Think、Ornith-1.0\u002F1.5-9B、Nanbeige4.2-3B、InclusionAI Ling-3.0-tiny、OpenBMB MiniCPM5-1B、NVIDIA Nemotron Nano 9B v2、AI9Stars G9v3 3B,以及 Prism ML 的 1-bit Bonsai-27B 和 Ternary-Bonsai-27B 两个极低比特模型([Liquid AI 8\u002F24 公告](https:\u002F\u002Fliquid.ai\u002Fblog\u002Fpipette-on-device-ai-benchmarking-by-liquid-ai))。\n\n## 数据里的几个反直觉\n\n**MoE 在端侧不一定省内存。** LFM2.5-8B-A1B 是 MoE,每 token 只激活 1.5B 参数(总 8.5B)。在 Galaxy S26 Ultra、2048 输入 token、Q4_K_M 配置下,它解码比 Qwen3.5-4B 快 2.4 倍,比 Ministral-3-3B-Instruct-2512 快 2.6 倍。但峰值 RAM 仍有 5.29 GiB —— 因为所有专家权重都要占内存,激活稀疏不等于\"模型小\"。这条对所有把 MoE 当成\"小模型替代品\"部署的人都是个提醒。\n\n**速度与质量是两条独立轴。** MiniCPM5-1B 和 LFM2.5-1.2B-Instruct 都是 ~1B 级别,Q4_K_M 量化,在 iPhone 17 Pro 上 2048 输入 + 256 输出,MiniCPM5-1B 用时 3.47 秒,LFM2.5-1.2B-Instruct 用时 4.12 秒(MiniCPM 快 15.8%)。但同一批量化产物跑 MATH-500,LFM2.5-1.2B-Instruct 高 9.0 分。两个轴都没有支配另一条。这意味着部署时必须明确优先级:是延迟敏感(选 MiniCPM),还是数学能力敏感(选 LFM),没有银弹。\n\n**架构微调比参数大小更影响长上下文行为。** Granite-4.0-H-350M 和 Granite-4.0-350M 同样 350M 参数、同样 Q4_K_M,在 Galaxy S26 Ultra 上从 256 输入扩展到 4096 输入时,H 版本保持 78.4% 的 decode 吞吐,普通版只保 33.8%。架构层面多花的设计,比单纯缩参数更能扛上下文压力。这给\"参数越大越好\"的常规直觉又添了一个反例。\n\n**两个跑分接近的模型,在不同任务上可能完全反转。** Granite-4.1-8B 和 Ministral-3-8B-Instruct-2512,Q4_K_M、在 M5 Max 上 2048 输入,decode 吞吐差 2.4%,峰值 RAM 差 1.2%。但同样的量化产物跑 IFBench,Granite 高 7.3 分;跑 GPQA Diamond,Ministral 反过来高 14.0 分。换句话说,跑分接近不代表可以互替 —— 任务分布决定哪个更合适。\n\n## 所以这事的意义\n\nPipette 把\"模型卡分数\"和\"在你手机上的实际表现\"之间的鸿沟,用一个公开、可复现的评测盖住了。三件以前很难说清的事现在有数据:同一参数量的不同架构在不同上下文长度下退化速度差别巨大;MoE 在端侧的\"小激活大内存\"特征会击穿\"参数越少越省\"的常识;量化后的质量损失高度任务相关,不能用一个 benchmark 推全场。Liquid AI CEO Ramin Hasani 的官方说法是\"on-device behavior is a property of the deployed system, not the model in isolation\" —— 这次的基准把这个判断变成了可量化的东西。\n\n接下来半年的行业动作可能会围绕这份榜单展开:AMD 的 Ryzen AI Max+ 395 与 Radeon 8060S 还没在榜单上,NPU 路径也因为各家 kernel 覆盖度不一致暂时没纳入对比。可以预见的是,端侧 LLM 的选型逻辑会从\"看 QPS 跑分表\"转向\"在目标设备、目标量化、目标上下文长度下的 Pareto 曲线\",这是这次发布给整个行业留的最实在的东西。","https:\u002F\u002Fartificialanalysis.ai\u002Fhardware-inference-stack\u002Fmobile-phones","3d6c2ae2-a449-467d-91dd-68fbbd04d714",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"fca9258a-9430-455a-b95d-b9fae5e373a8","ai-inference",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"120fa59a-ff6f-4537-9bf5-f818df636a0e","benchmark",{"id":19,"name":20,"slug":20,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":22,"name":23,"slug":23,"description":14,"color":14},"b49648f9-963e-4082-8684-3d085b7358fe","quantization",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"f46d9bc1-bfc7-48bb-8176-69eb96bacf40","en","Artificial Analysis turns the iPhone 17 Pro into the first open leaderboard for 23 on-device LLMs","Artificial Analysis and Liquid AI have launched the first open phone-side LLM benchmark, testing 23 quantized models that fit inside 8 GB. Headline finding: LFM2.5-8B decodes 2.4× faster than Qwen3.5-4B on the same device, but still peaks at 5.29 GiB of RAM because MoE expert weights must all live in memory.","## Background: Small LLMs on phones, but no open ledger\n\nFor the past two years, every small-model team (LFM2, Qwen3.5, Granite-4, and others) has been telling the same story: their model can deliver usable quality on a phone or laptop. The actual gap between \"runs\" and \"runs well,\" though, has never had a public ledger. Model cards ship fp16 benchmark scores; device vendors ship marketing videos. The independent middle layer was missing.\n\nArtificial Analysis, partnering with Liquid AI, has now published **Pipette**, a benchmark that pulls apart the deployment stack into five variables — model, quantization, runtime, device, workload — and measures them in the open ([artificialanalysis.ai](https:\u002F\u002Fartificialanalysis.ai\u002Fhardware-inference-stack\u002Fmobile-phones)).\n\nThe timing matters. Late August, Google told Android developers to cut memory usage by February 2027 because AI data centers are soaking up DRAM. Microsoft reported an internal engineer burning $28,000 of AI in 28 days. Both stories point to the same pressure: as cloud inference gets more expensive, demand for \"can the inference move to the device?\" is real. But device inference is not \"install an APK and you're done\" — quantization, runtime, KV cache budget, and context length each decide whether a model actually works on your phone.\n\n## How the benchmark works\n\nEligible models are those that fit in 8 GB after quantization, including the KV cache at 8K context. Tests use Q4_K_M or smaller GGUF quantizations, served through llama.cpp. Devices covered at launch: iPhone 17 Pro (12 GB), Galaxy S26 Ultra (12 GB), MacBook Pro M5 Max, with AMD Ryzen AI Max+ 395 + Radeon 8060S coming soon. The intelligence suite averages five evaluations with equal weight: BFCL (640-task tool-calling subset), IFBench (294 prompts), AA-Omniscience (6,000 questions on factual knowledge + hallucination resistance), GPQA Diamond (198 graduate-level science questions), MATH-500 (500 math problems). Inference is measured at a fixed 1,024-token prompt plus 256-token response, recording end-to-end latency, prefill and decode throughput, and peak RAM. Each evaluation runs five repeats at pass@1, and inference runs gate on a platform-specific thermal\u002Fidle check before timing to reduce environmental noise.\n\nThe first release covers 23 models: Liquid AI's LFM2.5 family, Qwen3.5\u002F3.6, Gemma 4 E2B\u002FE4B\u002F12B, IBM Granite 4.0\u002F4.1, Mistral Ministral 3, Llama 3.2 1B\u002F3B, TII Falcon-H1R-7B, Allen AI Olmo-3-7B-Think, Ornith-1.0\u002F1.5-9B, Nanbeige4.2-3B, InclusionAI Ling-3.0-tiny, OpenBMB MiniCPM5-1B, NVIDIA Nemotron Nano 9B v2, AI9Stars G9v3 3B, plus Prism ML's 1-bit Bonsai-27B and Ternary-Bonsai-27B as the extreme low-bit cases ([Liquid AI announcement, 8\u002F24](https:\u002F\u002Fliquid.ai\u002Fblog\u002Fpipette-on-device-ai-benchmarking-by-liquid-ai)).\n\n## Four counterintuitive findings\n\n**MoE on a phone does not necessarily save memory.** LFM2.5-8B-A1B is a Mixture-of-Experts model; only 1.5B of its 8.5B parameters activate per token. On the Galaxy S26 Ultra at 2,048 input tokens and Q4_K_M, it decodes 2.4× faster than Qwen3.5-4B and 2.6× faster than Ministral-3-3B-Instruct-2512. But its peak RAM still hits 5.29 GiB. Every expert weight has to live in memory, so \"sparse activation\" does not equal \"small model.\" This is worth remembering whenever anyone pitches MoE as a small-model substitute.\n\n**Speed and quality are independent axes.** MiniCPM5-1B and LFM2.5-1.2B-Instruct are both in the ~1B range, quantized to Q4_K_M. On the iPhone 17 Pro at 2,048 input + 256 output, MiniCPM5-1B finishes in 3.47 seconds versus 4.12 seconds for LFM2.5-1.2B-Instruct — MiniCPM is 15.8% faster. But the same quantized artifacts run on MATH-500, and LFM2.5-1.2B-Instruct scores 9.0 points higher. Neither dominates the other. Deployment now requires an explicit priority: latency-sensitive (MiniCPM) or math-sensitive (LFM). No silver bullet.\n\n**Architectural refinements matter more than parameter count for long-context behavior.** Granite-4.0-H-350M and Granite-4.0-350M share the same 350M parameter count and Q4_K_M quantization. On the Galaxy S26 Ultra, scaling input from 256 to 4,096 tokens, the H variant keeps 78.4% of its decode throughput; the regular variant keeps only 33.8%. The extra architecture work buys more long-context headroom than shrinking parameters does.\n\n**Two models with nearly identical throughput can flip rankings depending on the task.** Granite-4.1-8B and Ministral-3-8B-Instruct-2512, both at Q4_K_M, run on the M5 Max at 2,048 input tokens with only 2.4% difference in decode throughput and 1.2% difference in peak RAM. But on IFBench, Granite leads by 7.3 points; on GPQA Diamond, Ministral leads by 14.0 points. Similar numbers in the inference column do not mean interchangeable models in production — the workload distribution picks the winner.\n\n## Why this matters\n\nPipette covers the gap between \"model card scores\" and \"what actually happens on your phone\" with an open, reproducible methodology. Three things that were previously hard to argue now have data: same-parameter architectures degrade at very different speeds as context grows; MoE's \"small activation, large memory\" pattern on devices punctures the intuition that \"fewer parameters means less\"; and quantization-induced quality loss is highly task-dependent — you cannot extrapolate from one benchmark to the whole field. Liquid AI CEO Ramin Hasani's framing — \"on-device behavior is a property of the deployed system, not the model in isolation\" — has now been turned into something you can measure.\n\nThe next six months of industry moves will likely orbit this leaderboard. AMD's Ryzen AI Max+ 395 and Radeon 8060S are not yet on it; NPU paths are excluded for now because per-vendor kernel coverage isn't consistent. Expect the device-LLM selection logic to shift from \"check the QPS benchmark table\" to \"find the Pareto curve for your device, your quantization, your target context length.\" That is the most concrete thing this release gives the whole industry.","artificial-analysis-pipette-mobile-llm-benchmark","2026-09-01T11:00:00Z","2026-09-01T03:16:34.457860Z","2026-09-01T03:16:34.457868Z",true,"agent",130,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"218e966c-1521-44eb-87d1-dba77cc26c9c","CyberGym 的 86.3%：GLM-5.2 安全 Agent 开始用证据说话","cybergym-glm-52-evidence-agent","2026-07-29T06:33:39+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"aa53081a-e448-4087-aaa9-c822a7074bbc","LlamaWeb：WebGPU 跑 llama.cpp，16 设备吞吐 +45-69%","llamaweb-webgpu-llm","2026-06-29T18:00:00+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"43a0972c-f040-483c-88cd-89b40a22723d","RA-RFT：检索增强强化微调，让 LLM 学会「类比推理」而非「按字找参考」","ra-rft-retrieval-augmented-analogical-rl","2026-06-12T18:30:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"645dd52f-ae6e-4931-9d74-4582feb4ecb1","Google TurboQuant：LLM推理内存压缩6倍的技术突破","google-turboquant-kv-cache-compression","2026-04-23T01:11:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"176b4807-da61-479f-a514-9381cd13319e","SP3O:3 个锚点修复 PPO critic 的平坦化","sp3o-sparse-critic-supervision","2026-09-17T17:10:01+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"cb1e799d-d6d7-4ab9-9eaf-bea0aa432b06","Mistral 模型进驻 Firefox:119B 开放权重模型驱动浏览器 AI 助手","mistral-small-4-firefox-smart-window","2026-09-16T17:07:00+00:00"]