Same model, same tasks, a completely different report card. The decision model JEV, acting as an agent in the ALFWorld household environment, completes only 42.9% of tasks; moved behind the curtain as a teacher for Qwen2.5-1.5B, it drives its student to 82.9%. The paper, submitted to arXiv on November 12 (2611.17167) by the ZJU-REAL group at Zhejiang University together with Baidu, takes the much-discussed decision-model category and tests it in three roles - actor, judge, and teacher. The conclusion is counterintuitive: a model that acts poorly can still teach well.
How the three roles are defined
Decision models are a recent category: given a question and a context, they return a typed output - a choice among options with a probability for each - instead of generated text, which makes them fast and cheap. The paper evaluates three dedicated decision models: JEV from TypeSafe AI, the open-weight Decider v1.2 from Perplexity, and GPT Decisions from OpenAI built on GPT-6 Luna. Three general LLMs serve as controls: Qwen3.8-27B used via prompting, the same model via constrained readout, and GPT-6 Luna via prompting. The two test environments, ALFWorld (household instructions) and WebShop (online shopping), share a property: tasks contain subgoals that must be completed in order - in 'put the heated mug on the table', the mug must be heated before it is placed.
The three roles split cleanly. The actor picks the next action at every state. The judge scores already-executed actions, along two axes that combine: what it sees (the trajectory prefix up to the action, or the full trajectory) and what it is asked (whether the task will succeed, or whether this action helped the goal). The teacher feeds those scores into the advantage term of GRPO training; the decision model stays frozen throughout and is never queried at test time.
Why actors fail
All six models fall over on the same kind of task. JEV scores 70.8% on tasks whose two subgoals can be done in either order, but at most 18.8% on tasks that require heating or cleaning before placement; Decider v1.2, overall actor score 63.6%, still gets only 18.8% on Heat tasks; GPT Luna manages 34.3%.
The models do know the required step. In a goal-masking experiment over 116 key states, JEV puts 63.8% of its probability on the easy subgoal - placing the mug directly - and only 14.6% on the step that must come first; remove the easy subgoal from the instruction, and the probability on the required step jumps to 77.6%. The knowledge is there; the gravity of 'do the easy thing first' pulls the probability away. Adding a dependency reminder to the prompt barely helps - success changes by -2.8% to +5.2% across the six models - and the authors suggest restructuring the goal into ordered subtasks instead.
Change the question, and the judge works
For the same models as judges, two design choices decide everything: what they see and what they are asked. Seeing the full trajectory, JEV scores milestone actions 0.36 higher than no-op actions; asked whether the task will eventually succeed, the gap collapses to 0.05 - every action in a trajectory then gets the same answer, effectively replaying the outcome reward. Mean scores for milestone, other, and no-op actions spread from 0.632, 0.604, and 0.577 under the success question to 0.659, 0.440, and 0.298 under the contribution question. The paper also finds that the two dedicated decision models with disclosed base LLMs judge more accurately than those base LLMs given the same question, at 28% to 43% lower cost per score.
As a teacher: the small model overtakes the big one
The full-trajectory-plus-contribution-question combination goes into GRPO: Qwen2.5-1.5B rises from 72.8% to 82.9% on ALFWorld and from 56.8% to 71.4% on WebShop; the success-question variant yields only 69.3% and 68.3%. The 1.5B student beats a 7B model trained with outcome rewards alone on both environments. The gains land exactly on the tasks the actor itself rarely completes: on tasks with ordered subgoals the policy reaches 86.8%, while JEV itself completes only 14.7% of them as an actor.
The bill is modest: JEV is queried through OpenRouter at 0.042 USD per million input tokens with zero output cost, and teacher queries for a training run cost between 22 and 30 USD. No per-step labels, no separately trained critic - a frozen decision model scoring each episode once it ends is enough.
Why it matters
The paper genuinely separates 'can the model do the task' from 'can the model supervise the task': pick supervisors by their judging record, and judges can be screened cheaply before any RL run. The broader point - a strong scoreboard does not mean reliable decisions - got independent confirmation the same week: SO-Arena, a benchmark from the National University of Singapore covering 21 System One models (the very term TypeSafe AI used when launching JEV) across 11 domains, finds the model at the top of its scoreboard ranks only seventh on decision reliability. Two unrelated papers point at the same thing: capability and reliability are two different axes for these models.
The authors state their own limits: two text environments only, at most 30 actions per trajectory, and online settings where only the prefix is visible are out of reach for hindsight judging. Code is open-sourced under Apache 2.0 on GitHub; the paper is at arXiv 2611.17167. One question to leave you with: in your agent pipeline, is the model doing the scoring being asked the right question?