Decision Models Get Another Open-Source Contender

The decision-model category — a narrow LLM alternative for structured judgments — is getting crowded. Cloudflare open-sourced Clef, Perplexity put pplx-decider-v1-27b on Hugging Face, and TypeSafe's closed-source Jev has long served as the JevBench baseline. On October 15, ten authors led by Tong Zhu and Zhe Xu at Shanghai AI Lab, together with Shanghai Jiao Tong University, Fudan University, and Nanyang Technological University, posted Intern-Decision on arXiv, pushing the category forward on two axes: multimodality and single-pass scoring.

Three Sizes, Qwen3.5 Backbone, One Architecture

Intern-Decision ships in three checkpoints — 0.8B, 2B, and 4B — all fine-tuned from matching Qwen3.5 backbones with the vision tower and projector frozen. Instead of emitting free-form text wrapped in JSON, the model reserves a placeholder token in the assistant skeleton for every decision field and reads logits off the native LM head at the position immediately before the placeholder, then runs a softmax over the legal candidate set to produce a calibrated probability. All requested fields are scored in a single causal forward pass; no iterative decoding, no answer feedback.

Single-Point Accuracy Beats Jev, Calibration Better Still

  • 90.02% unweighted average across seven evaluation suites, against 88.74% for Jev v1.13.0 on the same items.
  • On 111 public Jevbench-Hard items: Intern-Decision-4B answers 82/111 (73.87%). After temperature scaling, ECE drops from 0.132613 to 0.065267; Jev hits 80/111 with ECE 0.094685 on the same set.
  • On a 96-case known-distribution pilot, the calibrated 4B lowers Brier from 0.628 to 0.550 and ECE from 0.213 to 0.089; in five of six categories it beats Jev on both metrics.

Local Latency, Single-GPU Footprint

For a fixed 289-token request with three fields (one choice, one Boolean, one score), running BF16 with SDPA on a single RTX 4090: 0.8B and 2B average 33.28–33.98 ms; 4B averages 44.16 ms. The reciprocal rates correspond to roughly 22.6–30.1 serial requests per second. The context window tops out at 32K, the legal-candidate cap is 62 symbols, and the HTTP service permits up to 16 fields per request — those are interface limits rather than generalization ceilings.

A Confidence-Based Cascade Worth Copying

The most copyable engineering pattern isn't the headline number but the confidence-based cascade the paper studies end to end. Feeding the 111 Jevbench-Hard items to the 4B and routing low-confidence requests to Atria-Dawn-Preview:

  • 4B alone: 82/111, 73.87%
  • Cascade (42 routed to Atria): 97/111, 87.39% (+15 over 4B, –2 vs Atria-only)
  • Atria alone: 99/111, 89.19%
  • Estimated total time: 140.16 s for the cascade, 300.03 s measured for Atria alone

In other words, a small model handles obvious cases, a larger one rescues the rest, and the engineering pattern is spelled out in concrete numbers rather than hand-waved.

So What

Intern-Decision re-trains Qwen3.5 specifically for structured decisions, and the 4B already beats the closed-source Jev v1.13.0 on public benchmarks. Three things follow. One, the decision-model category is decisively moving toward open-source multimodal variants; TypeSafe's closed-source pricing will feel more pressure. Two, the Qwen family's attractiveness as a "decision-model post-training base" is reinforced again (Perplexity, Cloudflare, and Microsoft have all picked Qwen), so the next wave of agent-targeted fine-tuning will likely keep landing on Qwen3.5. Three, the autoregressive masked-language-modeling + single-forward multi-field scoring recipe is portable to any open-weight model with a native LM head — the training data and objective function are the real moat.