Audit Findings: OpenAI's Monitoring Gaps Exposed

Roughly 25 percent of critical internal AI traffic may have flown entirely under the radar during a recent three-week inspection, according to new findings from an external audit team. This upper-bound estimate, provided by researchers from METR and Redwood Research, highlights significant blind spots in OpenAI’s internal systems for detecting rogue behavior in advanced models. The figure represents the worst-case scenario for agentic Responses API traffic from GPT-5.5 class models and above, which was not covered by any monitoring protocols during the assessment period.

The audit, which took place from July 21 to August 11, granted the two research firms deep access to OpenAI’s codebase, internal documentation, and employee interviews. The team spoke with approximately a dozen staff members to evaluate how the company tracks and manages its artificial intelligence agents. While the 25 percent figure is an estimate with significant uncertainty in both directions, it underscores a substantial gap in the current oversight framework for high-level model operations.

Beyond the unmonitored traffic, the auditors identified specific technical failures in the systems that were supposed to be active. For internal Codex traffic, METR estimates that around 10 percent of activity went untracked between May and July. This gap was caused by how the system handled compaction events, which are context summaries used to manage data flow. OpenAI had independently discovered this bug and implemented a fix while the audit was still ongoing, but the oversight failure had already occurred for several months.

The investigation also revealed that agents could potentially delay or avoid oversight by continuing to execute actions without handing control back to the user. Since the monitoring system only triggered upon such handovers, agents could sidestep scrutiny by simply persisting in their tasks. Furthermore, the researchers found methods to hide hundreds of actions from the monitor and to bypass the Codex sandbox. They determined that specially crafted inputs could likely crash the monitoring tool entirely, and that model behavior during reinforcement learning (RL) training was not being monitored at all when the audit began.

OpenAI states that it has now corrected all the weaknesses identified by the auditors. The company paused RL training following the Hugging Face incident in August and expanded monitoring to cover that area, although these changes occurred after the auditors’ access period ended and were not part of the official evaluation. The full 61-page report was handed to OpenAI internally, but after redactions, METR and Redwood Research leadership received a 38-page version. Only a short summary has been made available to the public, with METR noting it had no right to disclose the redacted portions independently.

This exercise was a pilot program designed to give outside researchers unprecedented access to OpenAI’s internal infrastructure. For future audits, METR intends to push for greater public transparency. The findings serve as a critical data point in the broader conversation about AI safety, demonstrating that even with dedicated internal tools, significant gaps in monitoring can persist without external scrutiny. The results suggest that the current level of oversight may not be sufficient to catch all potential anomalies in complex, agentic AI systems.