Agent training data is moving from “standard answers” toward the execution scene itself. NeoHorse-1 does not simply add another collection of static question-answer pairs. It records how a model writes code, calls tools, receives environmental feedback, retries, and eventually succeeds or fails. Those records are then used to decide what the next training round should learn.
The result: smaller models benefit from targeted post-training
TokenRhythm’s NeoHorse-1 comes in 4B and 9B versions, based on Qwen3.5-4B and Qwen3.5-9B. The paper evaluates agent execution, tool use, coding, and instruction following across ten benchmarks. The 4B model’s average rises from 58.94 for the same-size Qwen3.5-4B baseline to 64.87. The 9B model rises from 65.60 to 69.04. The important point is not a sudden jump on one leaderboard, but gains spread across several interactive tasks.
The table also does not show a uniformly positive story. On IFEval, the 9B model scores 89.09, below Qwen3.5-9B at 89.46. On LiveCodeBench v6, both score 65.14. Post-training is more visible on sustained operation, tool interaction, and failure recovery; gains on static instruction following are not uniform.
What is actually being trained
A conventional instruction-tuning example often pairs an input with an answer. An agent does not work that way. Its trajectory also contains tool calls, tool returns, environmental observations, retries, and an outcome. NeoHorse-1 organizes these records into user turns and subscenes, retaining reasoning, tool calls, and visible responses in the current turn. The model therefore learns not only what to say, but what action to take in a particular state.
The second piece is routing. The system is backed by a heterogeneous model pool rather than a single isolated model. The router records the estimated capability demand, the selected service tier, and the interaction that follows. The authors stress that the identity of the model actually served is not a simple proxy for task difficulty: user overrides, service availability, and deployment policy can all affect routing. NeoHorse-1 instead uses the router’s estimate of capability demand to arrange a three-stage curriculum. Lower-demand examples enter progressively, higher-demand interactions are added later, and some lower-demand coverage is retained.
That still leaves a mismatch. An offline trajectory contains a response that has already been produced, while deployment begins with prefixes generated by the student itself. NeoHorse-1 addresses this with routing-guided on-policy distillation. The student generates a response from a recorded context, and a teacher supervises the prefixes the student actually visits. Distillation therefore also responds to behavioral states exposed by the student.
Recursive self-improvement is currently a feedback-loop prototype
The strongest part of the paper is not its grand wording around RSI, but its decomposition into a checkable loop: routing and agent execution produce trajectories; the trajectories pass structural validation, six-dimensional semantic evaluation, and subscene labeling; capability feedback reallocates the next training mixture; the updated model returns to the harness.
“Self-improvement” here means using evidence from real execution to identify what the next training round should cover. The limitation is equally clear. The results demonstrate that one post-training cycle is effective; they do not demonstrate that a model can recursively become stronger without bound. The paper lists multi-iteration extension and broader task settings as next steps, so stability remains an open question.
The project releases 4B and 9B weights under Apache 2.0, together with code. The repository also provides BF16 and 8-bit, 5-bit, and 4-bit quantized versions. The native context length is 262,144 tokens, while the repository says the base capability can be extended up to 1,010,000 tokens. Deployment examples cover SGLang and vLLM, although practical capacity still depends on GPU memory and serving configuration.
My view is that NeoHorse-1 matters less because it has already fulfilled the fantasy of a model that upgrades itself, and more because it recycles what agent training usually wastes: failures, retries, tool returns, and routing decisions. The next model race may not be decided by whose static answer looks better, but by whose system uses the evidence left by every execution more completely.
Sources: arXiv:2609.08183; NeoHorse GitHub