UC Berkeley RDI recently published a paper exposing systemic vulnerabilities in current mainstream AI benchmarks. The research team built an AI agent capable of automatically analyzing benchmark code, auditing 13 widely used AI benchmarks. The result is alarming: every benchmark has exploitable security flaws, with 45 verified "cheat solutions" found in total.
The three most typical cases: First, Frontier-CS — submitted code and the evaluator run in the same Python process, and malicious code can replace the scoring function via stack-frame manipulation, directly achieving full marks. Second, Terminal-Bench — only needs to create a dummy C-extension file to pass existence checks, without actual invocation, to pass performance tests. Third, WebArena — the 812 tasks' reference answers are stored in files accessible to the agent; the agent only needs to read the JSON file to get full marks.
The common root of these vulnerabilities is insufficient isolation in evaluation infrastructure — the evaluated code shares memory or file system with the evaluation code. The paper points out a fundamental contradiction: benchmarks are not just tools for measuring model capability, they simultaneously shape model behavior. When benchmarks can be broken, models are effectively being trained to "cheat" rather than truly solve problems.
The impact is broad: benchmark scores determine model rankings, training-data pricing, and even some labs' release decisions. As major vendors are now required to pass security testing before release (the US Department of Commerce has included Google DeepMind, Microsoft, xAI in this requirement), the benchmark-credibility crisis is no longer just a technical issue, but has begun affecting release cadence.
A truly contamination-resistant evaluation method has yet to be established, and the industry's need for stricter evaluation practices is urgent.