The open-source LLM inference engine vLLM team released Semantic Router v0.3 (codename Themis) on June 5. The core idea is splitting the request into a five-layer explicitly auditable routing pipeline of "signal → projection → decision → algorithm → model," and for the first time introducing SAAR (Session-Aware Agentic Routing) to safeguard the model consistency of multi-turn agent sessions; with 350+ commits since v0.2, unifying local, Dashboard, Helm, and CRD configuration contracts, marking the LLM inference infrastructure moving from "model selection" to the "policy-as-code" operable stage.
vLLM's Themis release on June 5 is a rare "not model, not engine, but routing governance" version update in the open-source LLM inference ecosystem.
Themis does not pile on new features, but reassembles the router into five layers: Signal extracts evidence from requests, context, tools, and safety classifiers; Projection normalizes the evidence into readable policy bands like support_fast / support_balanced / support_escalated; Decision matches named policies by priority; Algorithm picks the model from candidates; finally lands on a specific model alias or provider. The entire contract is simultaneously dropped into the router, CLI, Dashboard, DSL, Helm Chart, and Operator, and the configuration fragmentation of the past "Docker one set, Helm one set, CRD one set" is completely converged.
More noteworthy is SAAR (Session-Aware Agentic Routing). In the past, single-turn routing only had to answer "which model should be used," whereas in a coding agent's multi-turn tool loop, tool results must return to the model that initiated the call, the provider-managed continuation id cannot cross physical backends, and the long-warm session cannot drop prefix locality because of the latest short request — these constraints now enter the routing policy layer, no longer left to each application to reinvent for itself. The supporting DSL synchronously introduces auditable structures like SIGNAL_GROUP, TEST, TIER, and EMIT retention, and the Dashboard's topology view and runtime metrics also follow the new surface, so policies can finally be reviewed like code.
My judgment: v0.3 is a "correct but not sexy" disruptive update — vllm-sr init is cut, config.yaml is upgraded to the authoritative file, unknown fields warn directly, and for teams still using the old configuration it's a migration cost, but in exchange routing drift is intercepted early, policies and configuration versions are traceable, and ops accidents can be replayed and reproduced. The competition in LLM inference infrastructure has finished the first half of "whose model is faster," and the second half has switched to "who can turn routing into an observable, auditable, replayable product" — vLLM uses Themis to give the first clear engineering answer.