[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-apple-mlx-muse-spark-mac-studio-thunderbolt-rdma":3,"topics-all":38,"news-related-36e4f28f-9176-485c-92b2-35cd0aac04fc":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"36e4f28f-9176-485c-92b2-35cd0aac04fc","Apple MLX 联合 Meta 把 816B 模型装进 Mac Studio 集群","Apple 与 Meta 合作,用 MLX 框架和 Thunderbolt 5 RDMA 把 816B 参数的 Muse Spark Open 拆到两台 M5 Ultra Mac Studio 上联合推理,四机集群生成速度超 36 tok\u002Fs。","把 816B 参数的 MoE 模型塞进两台桌面电脑,听起来像营销话术,但 Apple 这次拿出了技术细节。10 月 15 日,Apple Machine Learning Research 发布报告《Local and Distributed Large LLM Inference with MLX and RDMA on Mac Studio》(machinelearning.apple.com\u002Fresearch\u002Fmlx-distributed-inference),说明如何用 Thunderbolt 5 RDMA 把 Meta 同期开源的 Muse Spark Open 切成张量并行,跑在两台 M5 Ultra Mac Studio 上。\n\n## 一份被点名的实测报告\n\n报告给出 benchmark 命令:NVFP4 量化的 Muse Spark Open,prompt=8192、生成=128、batch=1,通过 mlx.launch 跑 mlx_lm.benchmark。四台 M5 Ultra Mac Studio 通过 Thunderbolt Mesh 两两直连,生成速度超 36 tok\u002Fs;相比 M3 Ultra,首 token 时间(TTFT)提速最高 4.6 倍。\n\n核心是 Apple 的开源数组框架 MLX。Mac Studio 统一内存让单台最多可用 512 GB,两台就是 1 TB,刚好装下 NVFP4 量化后的 816B 模型(每 token 激活 42B)。M5 Ultra 的 1.2 TB\u002Fs 内存带宽足以支撑单 token 张量分片读取。\n\n## RDMA over Thunderbolt 是关键一步\n\n传统分布式推理要么走 InfiniBand(贵、数据中心专属),要么走以太网(延迟高)。Apple 把消费级 Thunderbolt 5 的点对点能力做成 RDMA,本机线直连,延迟和带宽介于消费网和数据中心之间。Mac Studio 多口即插即用,两台机器变成「一台带 1 TB 统一内存的超算」。报告说这套架构不挑模型,是通用的消费级分布式推理底座。\n\n## Meta 为什么陪 Apple 站台\n\nMuse Spark Open 的卖点是 agentic coding 和长上下文。Meta 演示了 Muse Code 修改开源 iOS 应用 Ice Cubes、自动修 Bug、跑通完整构建,以及在 DaVinci Resolve 里做视频剪辑——AI 一帧一帧审素材、自动找最强镜头并竖裁。\n\n把整套管线搬上桌面,意味着开发者能在不连云的情况下跑出可商用的 agent 工作流。对 Meta 而言,这是给开源 816B 模型补了「本地能跑」的体验拼图;对 Apple 而言,这是把 M5 Ultra 统一内存的卖点从「跑 27B」拉到「跑 816B」。\n\n## 这条路通向哪里\n\n桌面级分布式推理不是新想法,但消费级硬件能做到 36 tok\u002Fs 跑 816B,门槛确实往下拉了一大截。报告说明 MLX 目标覆盖「27B 到 800B」本地推理全谱系。短期看,Mac 工作站用户不用租 GPU 也能跑前沿模型;中期看,如果 RDMA over Thunderbolt 被更多模型官方适配,本地化 agent 工作流可能成为开发者社区的新常态。这份报告真正的读者不是普通消费者,而是 Meta、Mistral、Qwen 这些开源模型团队的工程负责人——它在告诉他们:这堆权重怎么跑起来,我们已经铺好路了。","https:\u002F\u002Fmachinelearning.apple.com\u002Fresearch\u002Fmlx-distributed-inference","a2e6145a-2a88-4c51-8d09-c4375b2a833b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"fca9258a-9430-455a-b95d-b9fae5e373a8","ai-inference",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"40269b40-7942-4650-9672-ed2e6524d37a","ai-technology",{"id":19,"name":20,"slug":20,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":22,"name":23,"slug":23,"description":14,"color":14},"b9bd9039-fcdb-41a8-b85b-fc1587def2b9","open-source",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"ca888527-81e3-48fa-80d3-8e1186f46afb","en","Apple MLX and Meta put an 816B model on Mac Studio clusters","Apple's Machine Learning Research team published a real engineering report on October 15 showing how MLX and Thunderbolt 5 RDMA can tensor-parallelize Meta's newly open-weighted 816B-parameter Muse Spark Open across two M5 Ultra Mac Studios, hitting over 36 tok\u002Fs on a four-machine cluster.","Apple's Machine Learning Research team published a real engineering writeup on October 15 — not a press release — detailing how to fit a frontier 816B-parameter Mixture-of-Experts model into two desktop machines. The report, titled \"Local and Distributed Large LLM Inference with MLX and RDMA on Mac Studio,\" shows how to combine Thunderbolt 5 RDMA with Apple's open-source MLX array framework to tensor-parallelize Meta's newly open-weighted Muse Spark Open across two M5 Ultra Mac Studios.\n\n## A Benchmark Report That Names the Model\n\nThe report includes the actual benchmark command: NVFP4-quantized Muse Spark Open, prompt=8192, generation=128, batch=1, launched via mlx.launch running mlx_lm.benchmark. Four M5 Ultra Mac Studios connected pairwise through a Thunderbolt Mesh hit over 36 tokens per second of generation throughput. Compared to the previous-generation M3 Ultra, time-to-first-token improved by up to 4.6x.\n\nThe core is Apple's open-source MLX framework. Mac Studio unified memory lets a single machine expose up to 512 GB to the GPU; two machines give you 1 TB — enough to hold the entire NVFP4-quantized 816B model, which activates 42B parameters per token. M5 Ultra's 1.2 TB\u002Fs memory bandwidth is fast enough to stream the tensor slices each token needs.\n\n## RDMA over Thunderbolt Is the Key Move\n\nTraditional distributed inference runs on InfiniBand (expensive, datacenter-only) or Ethernet (high latency). Apple turned the peer-to-peer capability of consumer-grade Thunderbolt 5 into RDMA, letting two machines connect directly with cables — latency and bandwidth sit somewhere between a consumer network and a datacenter. Mac Studios ship with multiple Thunderbolt ports, so plugging in cables turns two machines into \"one supercomputer with 1 TB of unified memory.\" The report makes clear this stack is not Muse Spark Open-specific: it is a general-purpose, consumer-tier distributed inference substrate.\n\n## Why Meta Is Willing to Share the Stage with Apple\n\nMuse Spark Open's pitch is agentic coding and long-context workflows. Meta demonstrated Muse Code modifying the open-source iOS app Ice Cubes, auto-fixing bugs and shipping full builds, plus doing end-to-end video editing inside DaVinci Resolve — the AI reviews footage frame by frame to find the strongest moments and crop vertically.\n\nMoving the whole pipeline onto the desktop means developers can run production-grade agent workflows without a cloud round trip. For Meta, this fills in the \"actually runs locally\" experience for the open 816B model. For Apple, it stretches the M5 Ultra unified-memory pitch from \"runs a 27B model\" all the way up to \"runs an 816B model,\" and quietly writes MLX's real-world utility into a flagship demo.\n\n## Where This Road Leads\n\nDesktop distributed inference isn't a new idea, but getting 36 tok\u002Fs on an 816B model on consumer hardware does move the bar. The report benchmarks Mistral Medium 3.5 (128B) and Qwen 3.8 (27B) as well, signaling that MLX targets the full \"27B to 800B\" range of local inference. In the short term, Mac workstation users can run frontier models without renting GPUs; in the medium term, if RDMA over Thunderbolt gets official adoption from more model teams, local agent workflows may become the default for developer communities. The real audience for this report isn't consumers — it is the engineering leads at Meta, Mistral, and Qwen. The message: the weights are public, and we have already paved the road for how to run them.","apple-mlx-muse-spark-mac-studio-thunderbolt-rdma","2026-10-16T03:00:00Z","2026-10-16T07:07:59.440416Z","2026-10-16T07:07:59.440431Z",true,"agent",80,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"cb1e799d-d6d7-4ab9-9eaf-bea0aa432b06","Mistral 模型进驻 Firefox:119B 开放权重模型驱动浏览器 AI 助手","mistral-small-4-firefox-smart-window","2026-09-16T17:07:00+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"04ac054b-bc82-4832-9064-50a2defc33b2","Linus Torvalds 首批 LLM 辅助内核补丁进入审查","linus-torvalds-llm-assisted-folio-patches","2026-10-15T02:00:00+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"95e663a4-2cdf-454c-9787-b154fbd41909","TokenRouter:token 级路由提速 64 倍","tokenrouter-token-level-llm-routing","2026-10-09T21:11:28+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"ccce6dfe-776f-4cd8-9605-9163daea4627","Jeff 开源决策模型:2B 追平 Jev","jeff-open-decision-models-0-8b","2026-09-29T15:10:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"523d3ae4-9c60-4bcb-8e6c-ea98e58c7f71","一次前向一个决策:GLM-5.3-Flash 平替 Jev","glm-flash-single-token-decisions","2026-09-28T13:11:55+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"d78ea8a8-bebb-4b94-b340-127eb2874a73","Ovis 全模态嵌入 3B:综合分领先 5.19","ovis-omni-embedding-3b","2026-09-23T21:08:45+00:00"]