OpenAI released a new paper on "Deployment Simulation" — a method for predicting AI model behavior risks before launch by simulating real conversation traffic. The technique was used to evaluate GPT-5 before its public release, and the paper describes the methodology in detail.
The methodology: "deployment simulation" uses a large corpus of real user conversation traffic (with privacy-preserving anonymization) to simulate how the model will behave in production. The model is run on the simulated traffic, and a "risk monitor" checks every response for safety violations, bias, hallucinations, and other issues. The result is a "risk profile" that informs the launch decision.
The "pre-launch risk prediction" highlight: the deployment simulation is run continuously during the model development cycle, not just at the end. This allows the model team to catch risks early and iterate on mitigations. For GPT-5, the deployment simulation identified 12 high-risk behavior patterns, all of which were mitigated before launch.
The privacy-preserving aspect: the simulation uses anonymized, aggregated conversation patterns, not individual user data. The methodology is described in detail in the paper, and OpenAI has open-sourced the simulation framework for other developers to use.
The bigger takeaway: "pre-launch safety evaluation" is becoming a rigorous engineering discipline. The "we'll fix it in production" approach is no longer acceptable, and "deployment simulation" is a key technique for catching risks before users are affected. For the industry, this signals that "AI safety evaluation" is moving from "red team testing" to "continuous simulation," and the next round of investment in AI safety will include "simulation infrastructure."