July 14, arXiv:2607.12236 "Speculate with Memory" moves the memory system into the LLM Agent's speculative executor, giving "stateless speculation" online-learning capability. The core is adding three layers of serial online memory on top of the speculator: a contrastive transition table records historical action-action statistics, episodic memory retrieves past trajectory fragments similar to the current context, and a confusion tracker specifically suppresses repeatedly occurring errors. The three collaborate, giving the speculator, for the first time, the ability to "go through it once, be more accurate next time". The experiments cover three categories — action prediction, observation prediction, link prediction — on 6 benchmarks. Results: action-prediction accuracy improves 19–39% relative; on observation-prediction tasks with high action repetition, up to 2.5× absolute speedup is achieved. The paper especially emphasizes that all gains are "lossless" — speculation runs entirely in environment idle time, the actor trajectory is identical to non-speculative execution, with zero extra wall-clock overhead; and the gains continue to accumulate as memory grows, generalizing across speculators at different cost tiers. The real value lies in welding "specialized acceleration" and "continual learning" into the same pipeline. Current LLM Agent tool calls, environment observations, and action planning are tightly coupled, and latency stacks are very pronounced; most acceleration methods only focus on parameters or distillation. This paper's approach is essentially embedding "the veteran driver's muscle memory" into the inference loop — the same inference budget, getting better and better with use. This "diminishing marginal cost" acceleration path is more sustainable than simply scaling the model, and is closer to the real bottleneck of GPU utilization. The boundary is also clear: the more open the action space, the lower the memory hit rate — but Agents are increasingly moving toward open scenarios. The "lightweight memory + speculative decoding" combination offers all Agent-serving teams a low-cost, replicable engineering template; combined with vLLM and SGLang's existing speculative-decoding interfaces, it can be replicated quickly.