Writing a merger memo, building a financial model, putting together a due diligence report — none of that can be graded by a unit test. Whether an answer is good turns on judgment and taste, and neither regex parsers nor CI can verify it. Ultramarine 1 Flash, released by Paper Instruments on October 14, targets exactly that blank space: it post-trains GLM-5.3 Flash with large-scale reinforcement learning, treating "non-verifiable tasks" as the main battlefield where RL actually has to scale.

Building RL tasks the way verifiable ones are built

The core methodology in the report mirrors how recent verifiable RL work formalizes each task: an environment, a model instruction, and a verification system. The only difference is that the verification system has been swapped for a rubric.

For knowledge work, the deliverables are Excel models, PowerPoint decks, and Word or PDF documents. The evaluation pipeline parses and renders those agent-authored or agent-edited files into text, then hands the result to a naive LLM rubric grader — using the PerCriterionOneShotGrader from their open-source rubric package, paired with a custom grading prompt tuned for agreement with human experts following the evaluation process. On the internal PI Knowledge Work benchmark, the grader is GPT-5.6 Terra, running in the Feather agent framework with a 262,144-token context window.

The curriculum is cut finely: four domains (finance, law, consulting, operations), 19 professional practice areas, and 88 distinct environment types, with task instructions and rubrics adversarially generated along sampling axes. The report puts the combination space at 10⁷ to 10¹⁰.

Training configuration and sandbox isolation

The official report publishes unusually specific hyperparameters: CISPO with a modified two-policy objective, AdamW optimizer, batch size 64 × 8, LoRA rank 64, learning rate 5 × 10⁻⁵, sampling at temp = 1, top_p = 1, top_k = -1, and loss normalized by the total count of valid generation tokens in the training batch. The context window is 131,072 tokens, the agent runs up to 250 turns, and compaction triggers at 85% of the context window or when 30,000 tokens remain, whichever comes first.

The sandbox isolation section deserves its own look. The agent reaches application services through MCP, without direct access to the service processes or their storage; grading artifacts and verifier state sit outside the agent's execution boundary. The report puts it bluntly: nothing the model does in the sandbox can alter its own harness, credentials, or grader. Reusable verifier dependencies are baked into environment snapshots in advance, and task-specific grading material is supplied only after the rollout.

Results: knowledge work up, coding down

Ultramarine beats its base model on all six knowledge-work benchmarks, by 4.1 points on average, while generating 6.5% fewer output tokens. On JobBench it scores 62.9 against the base model's 56.3, with Gemini 3.8 Flash at 60.6 and DeepSeek V4.1 Flash at 60.4; on FrontierFinance it reaches 53.3.

But the coding benchmarks regress. Terminal-Bench 2.1 comes in at 83.2 versus the base model's 84.3, and DeepSWE 1.1 at 61.5 versus 63.4. On APEX-Agents 1.1 it scores 57.0, behind Claude Fable 5 at 63.6 and Gemini 3.8 Flash at 64.3.

Cost is the part that catches the eye. On the internal PI Knowledge Work benchmark, Ultramarine 1 Flash scores 59.3 at $0.13 per task. On the same harness, Claude Fable 5 (max reasoning) scores 57.8 at $8.19 per task and GPT-5.6 Sol (max) scores 55.1 at $2.09, while it stays within three points of Claude Opus 5.5 ($6.96 per task) and GPT-6 Sol ($1.26 per task). On that basis, CEO Daanish Khazi says it delivers "similar performance to Fable 5, while being 60x cheaper."

How to read these numbers

Three caveats matter.

First, every benchmark score originates from Paper Instruments. As of October 16, no third-party evaluation group has reproduced any of them, and the model is still tracked without an official rank on independent leaderboards.

Second, the 60x figure holds only for task-level runs on that internal suite — it is not a uniform token-rate difference. It uses each vendor's official API pricing as of October 2026, on tasks Paper Instruments constructed itself.

Third, the coding regression is genuinely informative. Post-training is not a pure gain: trained on knowledge-work rubrics, the model improves there while losing ground on software engineering, where verification signals are clean. Anyone hoping to claim that curve has to accept the cost written in the report's tables.

What is worth watching is not the 59.3 itself, but the mechanism behind it: treating rubrics as an RL verification signal, then using sandbox isolation to guarantee the agent cannot edit its own grading system. As "judgment work" becomes the default surface for agents, whoever operationalizes this approach first gets the migration-cost dividend.

The model is live on the API, and the company says weights will follow in about two weeks.

Reference: Paper Instruments technical report — https://www.paperinstruments.com/blog/ultramarine