Getting a large language model to write Metal kernels needs more than the model. It needs an execution environment where the model can compile, run, edit, and read back GPU timings, over and over.
The logic of on-device inference is straightforward: inference has to run on local accelerators, and Metal on Apple Silicon is the path that cannot be skipped. To teach a model to write Metal kernels you have to let it actually compile them on real hardware, run them, and look at the performance numbers. That naturally calls for several agent sessions running concurrently.
Then the trouble shows up. Using Apple Instruments Metal System Trace, the paper measures a single metal_nbody session with no competing agents: out of 31.6 minutes, the GPU is genuinely active for only 44.0 seconds, or 2.32% of the session. The rest of the time the agent is waiting on commands, editing code, and thinking.
Worse, concurrent sessions do not automatically fill the GPU. With sixteen sessions spread across four Macs, an average session spends 21.0 of its 45.1 minutes waiting on tools. Sessions on the busiest Mac wait 32.7 minutes on average, while those on the least busy wait 13.3 minutes. The cluster is already uneven internally.
Scheduling granularity drops to a single command
Metal Sandbox (arXiv:2611.17294, from Shanghai Jiaotong University, Beijing Zhongguancun Academy and StepFun) pushes the scheduling granularity down from one session to one command. Each agent gets its own sandbox instance, a dedicated workspace and a private HOME; commands run inside an Apple App Sandbox process. Anything matching an allowlist (file inspection, compilation, builds) executes in parallel, while everything else enters the single serial FIFO slot each node has.
The genuinely clever part is cross-machine migration. A serial command stuck in the queue can move to another Mac with a free slot through a reserve, claim and commit sequence, and its result is returned to the original agent. This requires every instance workspace and HOME to be mounted at the same absolute path across nodes (shared SMB mounts), while the controller drives each node CLI over SSH.
Two sets of numbers
The controlled experiments use four Mac mini M4 nodes (32 GiB memory each, macOS 27.0.1, connected through a gigabit Ethernet switch) running six hand-written Metal kernels. Under balanced submission, a 25.2 second batch on one node takes 6.62 seconds with static execution across four nodes, and 6.61 seconds with dynamic scheduling, so that gain comes from parallelism rather than migration. The real gap opens up under concentrated submission: static 25.3 seconds drops to 15.1 seconds, a 40.4% reduction and a 1.68 times throughput gain, while the within-batch queue p95 falls from 21.9 seconds to 11.9 seconds, down 45.5%. Longer commands do even better. At a six-second loop, 36.8 seconds becomes 18.0 seconds, throughput improves 2.05 times, and 588 commands per hour becomes 1,200.
The replay over real agent traces gives a different set of numbers. Sixteen mini-SWE-agent sessions (mini-SWE-agent 2.4.6, model Step 5 Preview, with 3,600 seconds of budget per agent and 600 seconds per tool) measure a batch of 67.6 minutes and mean tool waiting of 21.0 minutes, containing 114 parallel and 1,165 serial tool commands. Replaying those already-measured traces under dynamic scheduling drops mean waiting to 17.1 minutes (-18.8%) and the batch to 52.5 minutes (-22.3%), with the main replay completing 276 command migrations.
The bill is not that clean
Three things need saying.
First, migration is not cheap. Across 27 explicit migrations, looking up the command and reserving a slot takes a median of 0.818 seconds on its own; claim takes 0.97 milliseconds and commit 3.89 milliseconds. The whole path has medians of 1.86 seconds at zero load, rising to 2.16 seconds when reading 16 MiB. Short commands therefore lose money on migration, and the gains come mainly from long commands.
Second, this is redistribution, not extra compute conjured from nowhere. Of the sixteen replayed trajectories, eight wait less and eight wait more. The initially busiest Mac mini #A drops from 32.7 minutes to 15.8, while #C and #D incur more waiting. The paper notes that the replay fixes model behaviour and phase durations, so it cannot predict behaviour changes caused by different tool feedback. Across 166 parameter settings, the sensitivity sweep puts the mean waiting reduction between 6.78% and 20.4%, and the batch elapsed time reduction between 15.6% and 27.8%. The 18.8% and 22.3% headline numbers are one point in that band.
Third, the design itself breaks the premise of the measurement. The repository README states there is no GPU partitioning or GPU-call detection and no exclusivity, that other applications and parallel tasks may affect measurements, that file-access restrictions do not cover every application-launch path, and that this is trusted local development only with an unresolved LaunchServices escape risk. For a system whose whole purpose is measuring kernel performance, that means the numbers deserve a discount.
One note on provenance: this is an arXiv preprint submitted on 12 November 2026, running 16 pages with 7 figures and 5 tables, and the listing records no conference or journal acceptance. Readers should treat it as a preprint.
There is also no clean route to GPU passthrough on macOS: Docker's own documentation states that GPU passthrough to Linux containers is only available on Windows with the WSL 2 backend. That is part of why this paper skips the container route and returns to macOS native App Sandbox.
The mechanism is not a one-off finding
Needing a dedicated execution environment where a model can iterate on Metal kernels is not exclusive to this paper. Metal-Sci (arXiv:2605.09708, published at the Fifth Workshop on Deep Learning for Code at ICML 2026) drives evolutionary LLM kernel search over ten scientific compute tasks, reporting in-distribution self-speedups from 1.00 times to 10.7 times on M1 Pro. One of its cases makes the point sharply: the best GPT FFT3D solution wins in-distribution at 2.95 times speedup, then collapses to 0.23 times on a held-out cube of size 256, a silent regression the in-distribution score alone cannot see. The Dr. Kernel line (arXiv:2602.05885) fills in the same need from the reinforcement learning side, building a distributed GPU environment called KernelGYM to collect multi-turn interaction data.
Metal Sandbox and its Harbor integration are open-source on GitHub, at github.com/WangHaoZhe/metal-sandbox and github.com/WangHaoZhe/harbor.
So what
The value here is not a fresh benchmark number. It is pointing at a position that is easy to overlook: on-device inference wants Metal kernels, training a model to write Metal kernels wants many agent sessions, and those agents keep the GPU busy only about two percent of the time. The real bottleneck is often not whether there is enough compute, but how queuing and scheduling are written.
If you are building the training or evaluation environment for an on-device model, two judgements are worth copying directly. First, push the decision between parallel and serial down to the individual command rather than the whole session. Second, when queues inside a cluster are unbalanced, let waiting work travel to another machine. Just keep the three stated limits in mind: migration carries a fixed cost, the gains are fundamentally redistribution, and this environment does not guarantee GPU exclusivity, so any performance figure you measure with it deserves a second look.