[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-anemll-flash-ios-400b-moe-iphone-edge":3,"news-related-1abe59b8-844d-4bb1-bc27-cfe28099d101":36},{"id":4,"title":5,"summary":6,"content":6,"original_url":7,"source_id":8,"tags":9,"translations":23,"news_slug":29,"published_at":30,"created_at":31,"modified_at":32,"is_published":33,"publish_type":34,"image_url":13,"view_count":35},"1abe59b8-844d-4bb1-bc27-cfe28099d101","Anemll\u002FFlash-iOS：把 400B MoE 大模型塞进 iPhone 的端侧实验","Anemll\u002FFlash-iOS：把 400B MoE 大模型塞进 iPhone 的端侧实验\n\n2026 年 6 月 7 日，开源组织 Anemll 在 GitHub 上线了 **Flash-iOS** 仓库 —— 这是把 400B 参数级 MoE 大模型在 iPhone 上本地推理的项目首次有了\"完整 iOS 移植\"形态。它基于开发者 Alexintosh 的 FlashMoE-iOS 移植，叠加 iOS 内存特批、Fanout I\u002FO、Pread-only 路径等底层改造，让原本只能跑在 M5 Max 128GB MacBook 上的方案，第一次有了\"塞进 iPhone\"的可能。\n\n技术上有三个关键点。第一，**MoE 的稀疏激活**被推到极致。Flash-iOS 跑的是 Qwen3.5-397B-A17B：总参数 397B，但每个 token 只激活 17B，60 个 expert 层按需 SSD 流式分块预读，**纯 C\u002FObjective-C + 手写 Metal shader**，6GB 内存硬扛。第二，**Unsloth 的 3-bit 动态量化**首次集成到 MoE 路径：第 27 层 attention 保留 BF16，共享 expert 走 BF16，routed expert 走 Q5_K，其他层用 IQ3_XXS\u002FIQ4_XS 混合精度，让 5.44 MB\u002Fexpert 的体积换来接近 4-bit 的 PPL。第三，**iOS extended-virtual-addressing + increased-memory-limit** 与\"绕开 mmap\"的 pread-only 设计，是端侧能跑大模型的关键工程取舍。\n\n但冷静看，目前 0.6 tok\u002Fs 远未到实用程度 —— 这意味着每秒钟生成不到一个汉字。**真正意义在于：打破了\"模型权重必须全部驻留 DRAM\"的老约束**。当 LLM 参数规模继续向万亿推进，\"SSD 流式推理 + 3-bit 混合精度 + MoE 稀疏激活\" 这条组合路线，可能就是把千亿模型装进消费设备的唯一现实路径。Apple Silicon 的统一内存架构让这件事格外顺滑，国产开源大模型要复刻这条路径，工程链路的挑战远大于模型本身。","https:\u002F\u002Fgithub.com\u002FAnemll\u002FFlash-iOS","998df6db-96e6-4b8e-8be1-cfa00a6cd177",[10,14,17,20],{"id":11,"name":12,"slug":12,"description":13,"color":13},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",null,{"id":15,"name":16,"slug":16,"description":13,"color":13},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":18,"name":19,"slug":19,"description":13,"color":13},"b9bd9039-fcdb-41a8-b85b-fc1587def2b9","open-source",{"id":21,"name":22,"slug":22,"description":13,"color":13},"b49648f9-963e-4082-8684-3d085b7358fe","quantization",[24],{"id":25,"lang":26,"title":27,"summary":28,"content":13},"1cb47dac-3d96-41c6-8ce2-24e773f2eedf","en","Flash-iOS: an experiment stuffing a 400B MoE into iPhone","On June 7, 2026, the open-source organization Anemll put the Flash-iOS repository on GitHub — the first time a project of running a 400B-parameter-class MoE large model on an iPhone for local inference has taken on a \"complete iOS port\" form. It is based on developer Alexintosh's FlashMoE-iOS port, stacked with iOS memory entitlements, Fanout I\u002FO, pread-only paths, and other low-level modifications, allowing what was originally only runnable on an M5 Max 128GB MacBook to have a \"stuff into iPhone\" possibility for the first time.\n\nThree key technical points. First, the **sparse activation of MoE** is pushed to the extreme. Flash-iOS runs Qwen3.5-397B-A17B: 397B total parameters, but only 17B activated per token, 60 expert layers streamed by SSD on demand, **pure C\u002FObjective-C + handwritten Metal shader**, held down by 6GB of memory. Second, **Unsloth's 3-bit dynamic quantization** is integrated into the MoE path for the first time: layer 27 attention stays in BF16, shared expert stays in BF16, routed expert goes to Q5_K, and other layers use mixed precision of IQ3_XXS\u002FIQ4_XS, with 5.44 MB\u002Fexpert size in exchange for near-4-bit PPL. Third, the **iOS extended-virtual-addressing + increased-memory-limit** and the pread-only design \"around mmap\" are the key engineering trade-offs for on-device ability to run large models.\n\nBut staying cool, the current 0.6 tok\u002Fs is far from practical — that means less than one Chinese character per second is generated. **The real significance is: breaking the old constraint that \"model weights must all reside in DRAM.\"** As LLM parameter scale continues to push toward the trillion, the \"SSD streaming inference + 3-bit mixed precision + MoE sparse activation\" combination path may be the only realistic path to stuffing hundred-billion-parameter models into consumer devices. Apple Silicon's unified memory architecture makes this especially smooth, and for domestic open-source large models to replicate this path, the engineering chain's challenges are far greater than the model itself.","anemll-flash-ios-400b-moe-iphone-edge","2026-06-07T12:00:00Z","2026-06-07T12:09:57.046760Z","2026-08-19T02:08:40.142862Z",true,"agent",131,{"items":37},[38,43,48,53,58,63],{"id":39,"title":40,"news_slug":41,"published_at":42},"988bbfb8-672a-4c6c-98f7-3a170b6bd8b3","Macaw 把 LFM2.5 装进 1.5GB:4-bit 端侧 LLM 跑 Mac 控制工具链","macaw-lfm25-15gb-edge-mac-agent","2026-08-24T06:00:00+00:00",{"id":44,"title":45,"news_slug":46,"published_at":47},"d5203c93-2022-4769-a6d2-c7765ded2b40","腾讯混元 Hunyuan-A13B 开源实测:80B 总参 \u002F 13B 激活,GQA + FP8\u002FINT4 把 MoE 推理门槛打到消费卡","tencent-hunyuan-a13b-80b-13b-gqa-angelslim-moe","2026-07-30T06:00:00+00:00",{"id":49,"title":50,"news_slug":51,"published_at":52},"c2e18ecb-00b3-47ee-935a-f0e3cd0dee4a","Holo3.1 把 Computer Use Agent 拉进本地：FP8\u002FNVFP4\u002FQ4 三种量化让消费级 GPU 跑得动","holo3-1-h-company-computer-use-quant","2026-06-07T06:00:00+00:00",{"id":54,"title":55,"news_slug":56,"published_at":57},"cac485ab-a429-4ebc-88c6-a1f924f978ff","AWS 开源 KeysAndValues:微调时就让模型学会“遗忘”,单张 A100 撑住 128K","aws-keysvalues-sparse-attention-finetuning","2026-08-26T05:20:00+00:00",{"id":59,"title":60,"news_slug":61,"published_at":62},"48e1c261-a40a-4c71-9cba-450a459e6ad3","4-bit 模型反超全精度:QAH 把量化从性能税变成第二次蒸馏","quantization-aware-healing-hypernova-60b","2026-08-25T17:20:00+00:00",{"id":64,"title":65,"news_slug":66,"published_at":67},"c94bdf86-5de9-49fe-8c98-0f5c47611bfe","SGLang v0.5.18 发布:大模型冷启动提速 2.38 倍,710 个 PR 都改了什么","sglang-v0-5-18-cold-start-2-38x","2026-08-24T23:15:00+00:00"]