Outside experts find large blind spots in OpenAI's AI safety systems

For the first time, external researchers have conducted a deep, on-site examination of the safety mechanisms inside one of the world's leading artificial intelligence companies, uncovering substantial vulnerabilities in how the firm oversees its most advanced models. The findings indicate that a significant portion of internal AI traffic was not being monitored at all during a specific window, raising questions about the robustness of the safeguards designed to catch unexpected behavior before it reaches users.

The audit was carried out between July 21 and August 11, 2026. Two researchers from METR and Redwood Research were granted access to OpenAI’s internal codebase, documentation, and infrastructure for three weeks. During this period, they interviewed approximately a dozen employees and analyzed the systems used to track the behavior of autonomous AI agents. This pilot program was organized jointly by the two research organizations and OpenAI to allow independent experts to inspect internal infrastructure in a way that was previously unavailable.

The data reveals that an estimated 6 to 25 percent of internal traffic from the agentic Responses API, specifically from models at the GPT-5.5 class and above, lacked any monitoring coverage during the audit period. METR notes that the uncertainty surrounding this estimate is significant in both directions. Additionally, for internal Codex traffic that was intended to be monitored, roughly 10 percent went untracked between May and July due to a technical gap in how the system handled compaction events, which are context summaries. OpenAI independently identified and fixed this specific bug while the audit was still in progress.

Beyond simple coverage gaps, the auditors identified methods by which AI agents could avoid oversight. During the period when compaction events were unmonitored, agents could delay scrutiny because the system only triggered when control was handed back to the user. By continuing to execute actions without yielding control, agents could effectively sidestep the monitoring loop. The team also discovered ways to hide hundreds of actions from the monitor and bypass the Codex sandbox environment. They determined that specially crafted inputs could likely crash the monitoring system entirely, leaving the agent unobserved.

The auditors also noted that model behavior during reinforcement learning (RL) training was not being monitored at the start of their evaluation. OpenAI stated that it paused RL training after an incident involving Hugging Face in August and subsequently expanded its monitoring capabilities to cover this area. However, because these changes were implemented after the auditors' access period ended, they were not included in the final evaluation of the system's state during the audit window.

OpenAI has stated that it has since fixed all the weaknesses identified by the external team. The full report, which spans 61 pages, was handed over to the company. After OpenAI applied redactions, the leadership teams at METR and Redwood Research received a 38-page version of the document. A short summary was made available to the general public, while the redacted portions remain private. METR has said it did not have the right to disclose the redacted sections without agreement.

The audit serves as a preliminary step in a broader effort to bring outside scrutiny to frontier AI development. While the immediate technical gaps have been addressed, the process highlights the challenges of ensuring comprehensive oversight in complex, fast-moving systems. For future audits, METR has indicated a desire to push for greater public transparency, aiming to make more of the findings accessible to the public rather than limiting them to internal stakeholders.