Apple's Machine Learning Research team published a real engineering writeup on October 15 — not a press release — detailing how to fit a frontier 816B-parameter Mixture-of-Experts model into two desktop machines. The report, titled "Local and Distributed Large LLM Inference with MLX and RDMA on Mac Studio," shows how to combine Thunderbolt 5 RDMA with Apple's open-source MLX array framework to tensor-parallelize Meta's newly open-weighted Muse Spark Open across two M5 Ultra Mac Studios.

A Benchmark Report That Names the Model

The report includes the actual benchmark command: NVFP4-quantized Muse Spark Open, prompt=8192, generation=128, batch=1, launched via mlx.launch running mlx_lm.benchmark. Four M5 Ultra Mac Studios connected pairwise through a Thunderbolt Mesh hit over 36 tokens per second of generation throughput. Compared to the previous-generation M3 Ultra, time-to-first-token improved by up to 4.6x.

The core is Apple's open-source MLX framework. Mac Studio unified memory lets a single machine expose up to 512 GB to the GPU; two machines give you 1 TB — enough to hold the entire NVFP4-quantized 816B model, which activates 42B parameters per token. M5 Ultra's 1.2 TB/s memory bandwidth is fast enough to stream the tensor slices each token needs.

RDMA over Thunderbolt Is the Key Move

Traditional distributed inference runs on InfiniBand (expensive, datacenter-only) or Ethernet (high latency). Apple turned the peer-to-peer capability of consumer-grade Thunderbolt 5 into RDMA, letting two machines connect directly with cables — latency and bandwidth sit somewhere between a consumer network and a datacenter. Mac Studios ship with multiple Thunderbolt ports, so plugging in cables turns two machines into "one supercomputer with 1 TB of unified memory." The report makes clear this stack is not Muse Spark Open-specific: it is a general-purpose, consumer-tier distributed inference substrate.

Why Meta Is Willing to Share the Stage with Apple

Muse Spark Open's pitch is agentic coding and long-context workflows. Meta demonstrated Muse Code modifying the open-source iOS app Ice Cubes, auto-fixing bugs and shipping full builds, plus doing end-to-end video editing inside DaVinci Resolve — the AI reviews footage frame by frame to find the strongest moments and crop vertically.

Moving the whole pipeline onto the desktop means developers can run production-grade agent workflows without a cloud round trip. For Meta, this fills in the "actually runs locally" experience for the open 816B model. For Apple, it stretches the M5 Ultra unified-memory pitch from "runs a 27B model" all the way up to "runs an 816B model," and quietly writes MLX's real-world utility into a flagship demo.

Where This Road Leads

Desktop distributed inference isn't a new idea, but getting 36 tok/s on an 816B model on consumer hardware does move the bar. The report benchmarks Mistral Medium 3.5 (128B) and Qwen 3.8 (27B) as well, signaling that MLX targets the full "27B to 800B" range of local inference. In the short term, Mac workstation users can run frontier models without renting GPUs; in the medium term, if RDMA over Thunderbolt gets official adoption from more model teams, local agent workflows may become the default for developer communities. The real audience for this report isn't consumers — it is the engineering leads at Meta, Mistral, and Qwen. The message: the weights are public, and we have already paved the road for how to run them.