A quiet failure mode in supervised training for working agents is not data volume — it is broken tasks. One missing file or one inconsistent field can stop a teacher model from executing a professional task at all, and whatever demonstration survives carries the defect into the student model. WorkFactory, posted to arXiv on 27 October by the IQuest team, takes aim exactly there: treat the task itself as something that can be broken, diagnosed and repaired.

Give every task a health check first

The pipeline runs in three moves. Professional intents are composed from occupations, skills and real deliverables. Information-level planning then draws an information support graph that makes explicit which facts a deliverable depends on and where they must live, and the workspace is built against that graph. Clean probe executions test whether the task can actually be completed in the constructed environment; when support failures show up, targeted repairs fix the environment while preserving the intended work. Only environments that pass are retained, handed to teacher models for collecting independent demonstrations, and those trajectories then feed supervised fine-tuning. Controlled ablations in the paper show the planning-plus-repair loop reduces probe-diagnosed environment defects and improves downstream performance.

112,686 tasks across 612 occupations

The framework yields 112,686 tasks spanning 612 occupations and 23 occupational groups, with an average of 4.15 reference files per task. Teacher trajectories fine-tune Qwen models at 4B, 27B and 35B-A3B scales, producing the WorkPilot family.

Elo 879 to 1565 — and the 27B outruns the 35B

Across the four professional-work benchmarks reported in the paper — GDPval-AA, JobBench, Workspace-Bench and OmegaUse-OfficeVal — all three WorkPilot models beat their respective base models. WorkPilot-35B-A3B climbs from 879 to 1565 Elo on GDPval-AA v2.1, the 4B reaches 1375, and the 27B hits 1612 — actually ahead of the larger 35B-A3B. That lines up with the paper's scaling analysis: diminishing returns that depend on the model, with execution length first rising and then falling. One honesty note from the README: these Elo scores are local estimates calibrated to GDPval-AA v2.1, not an official leaderboard.

So what

The idea worth taking away is not another fine-tuned model but treating task executability as a measurable, repairable property — a check-then-repair-then-collect loop that can bolt onto any agent SFT pipeline. The dataset and all three models are already released, with the repository under Apache 2.0. Next time a release brags about "N million agent training tasks", ask one question first: can a teacher model actually finish them?

References: arXiv:2610.37161 · GitHub: IQuestLab/WorkFactory