[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-aa53081a-e448-4087-aaa9-c822a7074bbc":3},{"id":4,"title":5,"summary":6,"original_url":7,"source_id":8,"tags":9,"published_at":23,"created_at":24,"modified_at":25,"is_published":26,"publish_type":27,"image_url":13,"view_count":28},"aa53081a-e448-4087-aaa9-c822a7074bbc","浏览器里跑大模型不再\\\"挤牙膏\\\":LlamaWeb 用 WebGPU 把 llama.cpp 静态化,跨 16 个设备解码吞吐提升 45-69%","浏览器里跑大模型一直是个\\\"内存+跨设备兼容性\\\"双输的局面:不同浏览器、不同 GPU 上的内存占用差异巨大,加载动辄十几 GB 的模型更是家常便饭。arXiv 新论文《Llamas on the Web》(LlamaWeb)把 llama.cpp 后端重写成 WebGPU 原生实现,从根子上解决了这两个问题。LlamaWeb 的核心思路是\\\"静态化\\\":通过静态内存规划(Static Memory Planning)提前分配 WebGPU 缓冲,避免运行时反复申请释放带来的碎片;再通过可调内核库(Tunable Kernel Library)屏蔽不同设备 GPU 的差异,让同一份代码在 Apple、Intel、AMD、NVIDIA、Qualcomm 等 8 家厂商的 16 台设备上都能跑出接近原生性能。量化支持采用模板化 GPU 内核,允许扩展到任意新格式。实测数据很有说服力:相比现有浏览器 LLM 框架,内存占用降低 29-33%,在四款跨厂商 GPU 上解码吞吐提升 45-69%。在部分设备上,LlamaWeb 甚至击败了 llama.cpp 的厂商专用后端。这意味着开发者可以在 Chrome、Safari、Firefox 里直接集成 7B-13B 级别的 LLM,而不必依赖云端。对个人开发者而言,LlamaWeb 让\\\"完全本地、隐私优先\\\"的浏览器内 AI 应用有了真正的工程底座——不再因为内存墙而被迫阉割模型精度。配上 RAG 与 WebGPU 编译优化,这条\\\"零服务器 LLM\\\"的路径正在快速走向成熟。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2605.20706","7437aeb9-930c-4866-a2e9-48003c1a792b",[10,14,17,20],{"id":11,"name":12,"slug":12,"description":13,"color":13},"7ac06d8e-b074-4147-abfc-ffaa4c6b8744","ai-efficiency",null,{"id":15,"name":16,"slug":16,"description":13,"color":13},"fca9258a-9430-455a-b95d-b9fae5e373a8","ai-inference",{"id":18,"name":19,"slug":19,"description":13,"color":13},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":21,"name":22,"slug":22,"description":13,"color":13},"b49648f9-963e-4082-8684-3d085b7358fe","quantization","2026-06-29T18:00:00Z","2026-06-29T18:19:40.647308Z","2026-06-29T18:19:40.647323Z",true,"agent",4]