Autonomous ML research agents are usually judged on two things: the model backbone, and the harness — the surrounding engineering for planning, execution, memory, and verification. A paper posted to arXiv on Sep 2 by a BAAI team, Repo-To-Skill, argues there is a missing third layer: operational knowledge, the know-how that separates knowing a method from making it work. That knowledge is not absent from the field; it sits in repositories and papers, written for human readers and far too large to load mid-task. The paper's answer is to distill it into compact, verified, reusable skills.

DisCo: an agent that builds and uses its own skills

DisCo, the agent introduced in the paper, both creates skills and uses them during research. Distillation runs in two complementary forms: task-agnostic, condensing the field's widely used repositories into reusable skills, and task-oriented, producing the skills a concrete task calls for. The task-agnostic sweep across the open ecosystem produced the AREX-Skill Library: 5,000+ verified, executable skills distilled from 1,000 widely used ML repositories, organized into 20 research areas and 178 capability families. The library launched on Aug 3 covering 170+ repositories, and was extended on Aug 27 to 1,000 repositories and 5,000+ skills with a rebuilt router. The repository is Apache 2.0 licensed, though every skill carries its own license that must be checked individually.

The fixed-variable comparison

The key experiment holds the GPT-5.5 backbone, research harness, and downstream execution budget fixed, and toggles only whether the agent has the distilled skills. The GitHub README table (labeled Codex vs. Codex + AREX-Skill):

  • MLE-bench (75 Kaggle competitions, any-medal rate): 31.11% → 72.89%, a relative gain of 134.3%
  • PaperBench (20 paper replications): 29.45 → 39.59, +34.4%
  • FrontierCS (188 Agent Track tasks): 70.63 → 77.14, +9.2%
  • PassNet (200 compiler-pass samples, AS score): 1.343 → 1.531, +14.0%

The paper's explanation: skills supply reusable procedures, checks, and recovery paths that let the agent avoid expensive unguided trial-and-error and spend more of its budget on experiments and validation, with the advantage strongest on difficult tasks.

What a skill looks like

Each skill is organized around a SKILL.md with optional references/ and scripts/, capturing when a capability applies, what to run, how to validate it, and how to recover when an experiment fails. At runtime a router narrows a request to an area, family, repository, and workflow; the agent then loads only the branch it needs — progressive disclosure, so 5,000 skills do not blow up the context. The DisCo CLI ships as an npm package (v0.2.1, Node.js 22.19+), and skills can be imported into Codex, Claude Code, Pi, and other coding agents.

My take

The 134.3% figure is a relative gain from a 31.11% base, so read it calmly — but moving from a 31% to a 73% medal rate on a mature benchmark like MLE-bench is a real shift. The more interesting part is the path: no backbone change, no workflow change, just distilled operating context added under a fixed setup. Knowledge itself becomes a performance lever, which is a cheap route in a compute-anxious era. If the four-stage distillation pipeline — scope capabilities, ground evidence, construct the skill graph, verify and refine — replicates to other domains, two decades of GitHub repositories become mineable muscle memory for agents. One caveat: all four comparison numbers are self-reported by the team, and independent replication is welcome. Paper: https://arxiv.org/abs/2609.02749