Long Chain-of-Thought (CoT) is how reasoning models have been gaining score, but the cost is blunt: every extra thinking step burns tokens, and both the context window and the inference bill inflate together. For deployment teams, only a fraction of a several-hundred-token "thinking process" actually contributes to the answer. The question is which steps can be dropped. A paper submitted to arXiv on September 7 offers a distinctly geometric answer.

The method: a compass for the reasoning trajectory

The core observation is that CoT can be treated as a trajectory in hidden-state space. After projecting the representations of the question, each reasoning step, and the final solution into a 3D PCA space, the framework measures the angle between each step's local transition and the global question-to-solution direction.

A small angle means the step is heading straight for the answer — it stays as explicit text. A large angle means the step is checking, correcting, or exploring a branch — it gets compressed into a continuous latent token. The model outputs an interleaved explicit-implicit sequence instead of deleting or keeping the whole thought process. The paper also reports that the temporal variation of these angles reveals exploration, convergence, and refinement stages: small angles correspond to direct execution and answer formation, while large angles more frequently involve checking, correction, and branch exploration.

Training: two forcings, one for input and one for labels

To train this explicit-implicit architecture, the paper introduces two techniques:

  • Stepwise embedding forcing pools each redundant step into a single latent embedding;
  • Label forcing supervises that latent token with a soft multi-modal vocabulary distribution instead of a hard one-hot label.

Representation analyses show latent states form a compact region distinct from textual states, and the higher entropy at latent-token positions is interpreted as reflecting broader soft targets that encourage richer step-level feature learning.

The numbers: half the response, higher average accuracy

Experiments on Qwen3.5-9B and Qwen3.6-27B across six in-domain and out-of-domain benchmarks report:

  • average accuracy improved by up to 2.6%;
  • response length reduced by up to half;
  • Accuracy per Computation Unit increased by 2.29x;
  • preprocessing time reduced by 94.6% and training time by up to 80.3%.

Code is open-sourced on GitHub (AI9Stars/AStar-Thought); the paper is at arXiv:2609.07821.

How to read it

Three takeaways:

  1. Keep-what-matters beats prune-everything. Hard pruning loses intermediate information; keeping everything burns money. Using directional alignment as a continuous compression criterion is a clean, interpretable design — the "off-course angle" of each step is a computable quantity.
  2. The training-side numbers matter. A 94.6% cut in preprocessing time means the data-construction pipeline is lightweight in engineering terms, not a paper-only toy.
  3. Watch the "up to" phrasing. The 2.6% gain and the halved length are peak numbers; the distribution across the six benchmarks needs a closer read. Before production, stress-test on your own workloads.

For teams sensitive to inference cost, this latent-compression direction belongs on the watch list: when a model "thinking too much" becomes the bulk of the bill, a model that knows which step doesn't deserve explicit text is the money-saving model.