Long-horizon coding agents have an awkward cost structure: a single task easily runs dozens or hundreds of steps, and the common practice is to send every step to a frontier cloud model. An NVIDIA research team recently posted a paper, EdgeWise, that calls out the problem in its opening — not every step needs frontier-level capability, yet every step is billed at frontier prices.
Cloud Value Concentrates in a Few Key Steps
The paper starts with an observation study: the authors analyze 2.4k coding-agent trajectories and classify each step into six action types — Write/Edit, Read/Search, Diagnose/Reproduce, Test/Validate, Discover/Navigate, and Wait/Poll — then measure the value of sending each type to the cloud. Their conclusion: the benefit of the cloud model concentrates in a small subset of critical steps. With a cloud usage rate of only around 40%, delegating the right steps yields performance comparable to using the cloud model for the entire trajectory, while cutting inference cost by at least 25%.
Teaching a 9B Model When to Ask for Help
Based on that observation, EdgeWise deploys Qwen3.5-9B locally as the primary solver and hands control to a cloud LLM only at critical steps. How to decide which steps are critical? The paper offers two mechanisms. The first is an external judge: a lightweight critic model (GPT-5.4-mini) decides online whether the current step needs a stronger model. The second is self-delegation: a step-level RL method trains task solving and delegation jointly into the 9B model itself, so no extra decision model is needed at inference time. Training uses the SWE-Smith environment with Kimi-K3 as the cloud teacher: 2K examples for SFT, followed by 175 RL steps on 40 NVIDIA A100 80GB GPUs. The ablation is telling — the SFT-only model reaches 57.30% pass@1, while adding RL lifts it to 71.91% for just $3.74 more. The delegation policy, not the task skill, is the real delta of the RL stage.
The Numbers: Up to 62% Cheaper, and Half Again on the Edge
In the main results, EdgeWise reduces inference cost by up to 62% on SWE-Rebench and 34% on Terminal-Bench 2.1, averaging 46% across models and benchmarks, while matching or outperforming pure-LLM execution; a comparison with Claude Opus 4.8 as the cloud model is also included in the main table. The team further deploys Qwen3.5-9B on a single NVIDIA DGX Spark (vLLM 0.19.1, BF16, GPU memory utilization 0.65) for a real edge validation: Pass@3 stays comparable to the A100-based setup while cost drops roughly another 50% versus all-cloud execution. Generalization holds too — swapping the local model to Tmax-9B, binary delegation lifts pass@1 from 28.09% to 74.16%, tying GPT-5.5 at 74.16% while saving $27.67; replacing the judge with the open-source Qwen3.8-27B keeps performance comparable.
So What
The interesting part of this paper is not yet another "small models are cheaper" story — it turns "should I ask for help" itself into a trainable skill. For agent builders, the bulk of the API bill was never the hard problems; it was routine steps burning frontier-model tokens. For NVIDIA, the framework paves a natural road for devices like DGX Spark: a resident local 9B with on-demand cloud escalation may become the default shape of long-horizon agents. Paper: https://arxiv.org/abs/2610.17560