As LLM Agents start to automate complex workflows for you, the industry has long lacked an evaluation standard that can truly test their deep-reasoning capability. Most benchmarks test only local tasks, and can barely quantify multi-step dependencies and long-horizon state tracking.
Meituan's Changlu team and Fudan University recently jointly released AgentEscapeBench, an escape-room-style tool-reasoning evaluation benchmark. 270 tasks across 5 difficulty levels, with the core test: on a directed acyclic graph (DAG) structured tool-dependency chain, can the agent correctly infer the calling order, track the progressively revealed hidden state, and produce verifiable answers.
The experimental results reveal a thought-provoking fault line: the strongest model achieves 90% accuracy on shallow dependencies (difficulty 5), but accuracy plummets to 60% when depth rises to 25. Humans, on the other hand, slide slowly from 98.3% to 80% — this means current LLM Agents still have obvious generalization shortfalls in truly multi-step collaborative tool-calling scenarios.
The paper attributes failures to three capability degradations: long-horizon state tracking, instruction stickiness, and intermediate result passing. AgentEscapeBench supports fully automated evaluation, allowing the entire community to iterate quickly.
For engineers, the practical significance of this research is clear: current Agent frameworks cannot yet reliably replace humans when handling complex tasks with long dependency chains, and need to be equipped with more refined memory-management mechanisms to compensate for this shortcoming.