How Hard Does It Think? Step-Aware Reasoning Energy (SARE) Reads the Inner Workload of Each LLM CoT Step

We are increasingly used to letting LLMs externalize their thinking in chain-of-thought (CoT) traces, and then reading the final answer. But a natural question keeps being sidestepped: how much "effort" did the model actually spend on each step? Was effort spread evenly across the chain — or did a single step suddenly absorb most of the compute?

A new paper on arXiv — "How Hard Does It Think? Analyzing Step-Aware Reasoning Energy in LLM Chain-of-Thought Trajectories" (arXiv:2607.28674, Wei, Huang, Han, McAuley et al., submitted 28 July 2026) — does not produce another benchmark leaderboard. It turns the lens inward.

Why previous accounting was too coarse

The authors point out that earlier ways of "adding up" LLM reasoning are quite coarse:

  • Output-level signals — token logprobs, entropy, self-consistency — tell you only whether the answer looks plausible, not whether the reasoning behind it actually cost anything.
  • Trajectory-level scalars — trajectory entropy, reasoning consistency, and similar — compress the chain into a single number, hiding any sense of which step consumed compute.

The result is what the authors describe as "step-wise effort opaque" — exactly how much reasoning resource is consumed at each step remains a black box.

What SARE actually does

SARE stands for Step-Aware Reasoning Energy. It does not rely on output confidence at all. Instead, it uses a geometric measure: compute the Centered Kernel Alignment (CKA) between the Gram matrices of token hidden states in adjacent transformer layers. CKA captures the "shape similarity" between two representations without requiring eigenvector alignment or cluster correspondence — a smaller alignment cost than Procrustes / RSA-style alignment methods.

SARE then embeds this energy into the semantic progression of the CoT: the chain-of-thought is modeled as transitions among latent semantic states, so "how much energy was spent on a given step in a given state" becomes a directly observable quantity.

What the paper reports (six benchmarks, three open-weight LLMs)

The findings, in the paper's own framing:

  • Energy is highly non-uniform and shows phase-like transitions. Across step types, the energy varies sharply, and some adjacent steps display "phase-like" jumps — transitions that vanish in trajectory-level metrics.
  • Incorrect trajectories show systematically lower energy at critical reasoning junctions. When correct and incorrect traces are compared side by side, the incorrect ones dip in energy exactly at the critical junctures. In other words, "under-effort" at a key step correlates with getting the answer wrong — and this correlation lives inside the model, invisible from the outside.
  • SARE-based features match or beat output-confidence baselines in most settings. The paper's wording — "match or outperform in most settings" — is the literal qualifier we are preserving here; we are not rewriting it as "best in class" or "first to surpass."

The paper evaluates three open-weight LLMs across six reasoning benchmarks. The specific model and benchmark lists are reported in the paper itself, and we follow the abstract's framing rather than reconstruct the details.

How this differs from product-side "thinking budget" work

In industry, Anthropic has already turned thinking effort / medium effort into a product-level dial inside Claude. Its logic is causal in the direction input → budget → output: at the user-facing entry point, declare "I am willing to spend this much compute," and the model budgets accordingly.

SARE goes the opposite direction: it does not tell the model how to allocate effort — it reads, after the fact, how much effort the model's hidden states indicate. The two together are interesting:

  • The industry direction is budget control: how to spend less.
  • SARE's direction is budget audit: how to see what was actually spent.

If SARE-style methods are validated on larger models and more out-of-distribution tasks, three things may follow:

  1. Inference-time pricing by reasoning intensity becomes more credible — because today it has to lean on crude proxies like output length.
  2. Failure attribution gets a new axis. Failures may no longer collapse into "the model got the answer wrong"; they can be partitioned into cases where the model was under-efforting at the wrong moment — a target for the next round of training.
  3. A bridge to mechanistic interpretability. Combined with existing mech-interp work, SARE can gradually separate which CoT steps are actually doing math from which steps are performing the chain. That latter distinction is exactly what the Quanta long-read on LRM reasoning only flagged and could not quantify (see Quanta's "Is AI Reasoning Right for the Wrong Reasons?", aggregated by Solidot entry 84988). SARE offers a measurable angle.

Two honest limits

  • The paper reports a method and observations. It does not, on its own, demonstrate downstream applications such as "discard low-energy steps to save compute." That has to come from follow-up work.
  • "Match or outperform in most settings" is the paper's own qualifier. We deliberately do not replace it with broader phrasing like "broadly" or "across the board."

The interesting move here is not another benchmark number. It is that "is the model slacking during reasoning?" becomes a quantity you can read out of geometric measurements — something that, until recently, was mostly inferred from logs and human spot-checks.

Reference

arXiv:2607.28674, How Hard Does It Think? Analyzing Step-Aware Reasoning Energy in LLM Chain-of-Thought Trajectories, Wei et al., submitted 28 July 2026.
https://arxiv.org/abs/2607.28674