Anthropic's Claude Cracks the Code on Software Safety

For decades, proving that software is truly safe has been a luxury reserved for the most critical systems. It was so expensive and complex that a team of specialists spent twenty person-years just to verify the kernel of a small operating system. As a result, this rigorous method of formal verification was largely confined to high-stakes environments like aircraft avionics, leaving the vast majority of software to rely on standard testing that often misses subtle errors. Now, Anthropic claims to have changed that equation by using its Claude model to automate the entire proof process, effectively turning a multi-year human effort into a task that takes only weeks.

The core of this new capability lies in a collaboration between Anthropic’s Frontier Red Team and a partner called Theorem. Instead of writing new software from scratch, Claude was tasked with verifying existing code. The model drew on test programs from 123 open-source libraries within Google's OSS-Fuzz security project. In a matter of weeks, Claude generated approximately 180 million lines of mathematical proof code. This volume of output represents a significant leap in automation, as it demonstrates that an AI can now handle the heavy lifting of translating software behavior into mathematical certainty without constant human oversight.

The process works by having Claude translate crash tests into mathematical problems and analyzing the compiled machine code. Once the model produces these extensive proofs, an independent verification program named Rocq automatically checks them for validity. According to Anthropic, this workflow requires almost no human reviewers, which the researchers describe as "fully autonomous" proof generation. While the formal foundations and tools were built by humans at Theorem, the execution of the verification is now largely in the hands of the AI.

Perhaps the most surprising outcome of this experiment is the number of bugs Claude discovered. While generating proofs, the model encountered hundreds of flaws and was permitted to fix them itself. Many of these bugs would have slipped past conventional testing because they only manifest under very specific conditions, such as unusually large input files or a particular sequence of memory accesses. In one notable instance, the team found as many security bugs in a single week while verifying a widely used PDF reader as are typically reported for that software in an entire year. In another case, Claude identified a logical error that had remained undetected for over a decade.

This ability to find hidden defects suggests that even failed proof attempts have value. Anthropic notes that if a complete proof ultimately fails, the process still surfaces real bugs that might otherwise have gone unnoticed for years. This shifts the perspective on AI-generated software from a source of potential risk to a tool that can actively improve code quality by exposing weaknesses that human testers might miss.

However, there are strict practical limits to what this technology can do. Anthropic is clear that being "formally verified" does not mean the software is error-free in every possible scenario. The proofs only hold under specific conditions: a particular processor architecture, single-threaded programs, and inputs under 64 MB. In the real world, these libraries often run on different hardware and in more complex configurations. Therefore, this verification provides a strong guarantee within defined boundaries, but it is not a universal shield against all potential failures.

The cost of running these verifications is also significant. Individual proofs can require hundreds of millions of tokens to complete, and Anthropic has not reported the total project costs. Despite this, there is evidence that efficiency is improving. A smaller test run showed that proofs are reusable, and newer models are becoming more efficient. Specifically, the Opus 5.5 model solves half the required proofs with about 40 percent fewer tokens than its predecessor, Opus 5.

If AI can verify its own code, developers may be able to trust AI-generated software more and spend less time manually reviewing it. This could save costs not just in verification but across the entire coding process. The project strengthens the argument for broader software automation, shifting the industry question from "Can AI program?" to "Can AI prove that the code it writes is safe?" If the answer is yes, the implications could extend far beyond current coding tools.

Anthropic expects that formal verification with AI will soon become as routine as standard testing. Theorem’s roadmap already targets critical infrastructure components like glibc, OpenSSL, and cURL. While the technology is not yet ready for every type of software, it marks a major step toward a future where mathematical certainty is a standard feature of the development process rather than an expensive exception.