Production systems rarely run a single model for a whole conversation. A small model handles the easy requests, harder ones get routed to a large model, and long conversations may switch models mid-thread — cascading, routing, and model switching. The problem is that every switch forces the target model to re-run prefill over the context accumulated so far, and that redundant computation grows with context length. A paper posted to arXiv on November 5 tries to skip the repetition entirely: not by passing tokens, but by handing over inference state.
Beyond the K/V cache there are two more state types
Existing cross-model state transfer work mostly moves the K/V caches of full-attention layers. Cross-Model fits closed-form affine mappings, CacheBridge constrains support layers by attention sensitivity, and CacheLink adds a nonlinear correction trained through a frozen receiver.
Hybrid linear-attention models are not like that. Models such as Qwen3.5 interleave full-attention layers with Gated DeltaNet layers, and the latter keep two additional pieces of state: a recurrent state matrix (REC) and a convolution history (Conv). Both are fixed in size and do not grow with prefix length, which is exactly what makes long-context inference cheap on memory.
The trouble is the coupling. Conv feeds into the keys and values produced for recurrent layers, and those signals in turn drive REC updates. Conv history slots get overwritten by later tokens, but whatever was written into REC lingers.
More accurate reconstruction, worse continuation
The paper runs a set of controlled interventions, and the conclusions are distinctly counterintuitive. In native Qwen3.5-9B, only Conv is zeroed; at step 8, restoring native REC gives an output KL of 0.00721 while restoring native Conv gives 0.07023 — nearly a tenfold gap. A perturbation written into REC therefore persists past the convolution window and keeps affecting later output.
A second result is sharper. A residual term added specifically to reconstruct REC more accurately drives the initial relative MSE down from 1.00671 to 0.28380, yet the full-model continuation KL goes the wrong way, from 0.64829 up to 0.69142. A different residual candidate pushes MSE to 0.24696 with a KL of 0.67841 — again worse than the baseline. Local reconstruction error and downstream usability are simply not the same thing.
A native-state replacement study points the same way. Across 18 documents, swapping the transferred state for native state drops composite KL from 0.59291 to 0.01664. Replacing REC only gives 0.10060, FA only gives 0.33515, and replacing Conv only gives 0.61845 — worse than replacing nothing at all.
How HST does it
The approach combines structural mapping with functional calibration. All three state types move together — the FA K/V caches, the REC matrices, and the Conv histories — and are installed at a single prefix boundary along with position metadata. REC uses group-wise affine mappings fitted separately per model pair; the 4B→27B configuration adds a correction conditioned on the currently mapped state, with rank at most two. Conv is aligned with affine channel maps that strictly preserve history order, because that order is itself part of the convolution's semantics. On the FA side, increments follow a low-rank LoRA-style parameterization and are merged offline back into the base mapping weights, so online handoff still applies a single affine transform. Both models' pretrained weights stay frozen throughout.
Calibration is the whole point of the paper: rather than approximating what the source state looks like, the mapping parameters are inferred from the frozen target model's outputs during continuation. The authors call this continuation compatibility and stress that it is a separate criterion from reconstruction error.
The numbers
The benchmarks cover ARC-C, HellaSwag, WinoGrande, MMLU, and GSM8K. Mean task retention for 4B→27B and 9B→27B is 89.88% and 94.11% respectively, which the paper reports as 3.56 and 7.10 percentage points above the strongest compared baseline, CacheLink. Most of the 9B→27B gap comes from MMLU, rising from 67.93% to 88.22%. GSM8K at 4B→27B is the dramatic one: CacheBridge manages only 5.74%, while HST reaches 91.66%.
Latency is measured separately on synthetic 16K inputs, with source caches already available, and excludes source prefill, the vocabulary output projection, and answer generation. For 9B→27B, the time falls from 8823.37 ms to 402.79 ms, a 21.91x speedup; for 4B→27B it drops from 8644.23 to 399.02 ms, 21.66x. Short contexts gain far less: at 64 tokens it is only 3.01x and 2.78x. The ablation shows the FA increment doing the heavy lifting — with both modules off, GSM8K is 63.08%; REC correction alone gives 66.57%; FA increment alone gives 86.73%; both on gives 88.32%, against 96.29% for native 27B. Calibration is not free either: the FA stage adds 0.655M parameters in each direction, trains for 768 updates, and takes 2.92 and 3.36 hours.
Lineage and limits
This line of work did not start with this paper. LatentPort, in September, demonstrated cross-model handoff of recurrent state on a Qwen3.5 4B→9B pair and described itself as the first such demonstration, but covered only one direction with 4K teacher-forced continuation, and its near-native gate failed. Cross-Model KV Cache Transfer, from August, gives a closed-form solution for in-family K/V migration, reporting 73% to 98% fidelity and 2.7x to 25x speedups on four matched pairs, and explicitly names hybrid architectures carrying SSM state alongside KV as the natural next step (see arXiv:2609.25053 and arXiv:2608.03893). HST's contribution is pulling Conv into the picture, scaling the target up to 27B, and putting fidelity and latency on the same table for comparison.
The paper draws its own boundaries: it validates only on the Gated DeltaNet family of Qwen3.5, with transfer to other architectures such as KDA and to MoE models untested. Fidelity is measured on benchmarks while latency is measured on synthetic contexts, so the quality-efficiency trade-off has not been jointly evaluated under one consistent inference setup.
My take
The 21.91x figure deserves a discount: it assumes the source cache is already computed, and it excludes answer generation. In a real routing system, "when should we switch models" and "how much does switching actually save" are two separate decision problems, and the paper does not address how to wire them together.
But what is genuinely worth carrying away is not the multiplier — it is the finding that more accurate reconstruction does not mean better continuation. In that REC comparison, reconstruction error fell by roughly 72% while continuation KL rose by about 6.7%. That is close to the trap every state compression, quantization, and distillation effort walks into: the proxy metric looks great while end-to-end behaviour quietly degrades. Anyone optimizing against a proxy metric deserves to have that number shown to them.
So what now? If your service already does model cascading or routing, the thing worth trying next time you switch is not just "have the small model summarize the context and pass that along" but handing the state over directly — the full paper is at arXiv:2611.08133.