Anthropic and Theorem use AI to verify open-source code security
Anthropic and Theorem demonstrated that large language models can perform formal software verification at scale. Using Claude, the teams proved security properties in over 100 real-world open-source libraries at the RISC-V assembly level. This approach uncovered long-standing bugs missed by extensive testing, suggesting AI could soon make rigorous code guarantees practical for widespread software development, significantly enhancing cybersecurity against advanced AI-driven threats.
Key points
- Anthropic, the developer of the Claude AI model, collaborated with Theorem to apply large language models to formal software verification.
- The team used Claude to prove security properties at the RISC-V assembly code level across more than 100 real-world open-source software libraries.
- This effort represents the first large-scale application of AI to formal verification of production software at the assembly level.
- During the proof-writing process, Claude identified bugs that had persisted for years despite trillions of previous test runs.
- The collaboration aims to address the growing inadequacy of standard defenses against cyberattacks, which are increasingly exploited by AI.
- The researchers anticipate that formal verification will become an essential component of secure software development as model capabilities improve.
Anthropic has announced a significant advancement in software security through a collaboration with Theorem, demonstrating that large language models can now perform formal software verification at a practical scale. The joint effort focused on proving security properties within real-world open-source software, a task that historically required years of specialized expert effort and was often impractical for most development projects.
Using the Claude AI model, the team successfully verified security at the RISC-V assembly code level across more than 100 open-source libraries. This marks the first large-scale effort to apply AI-driven formal verification to production software at such a low level of abstraction. The process not only established rigorous security guarantees but also revealed latent bugs that had evaded detection for years, despite the software undergoing trillions of test runs.
The initiative addresses a critical gap in modern cybersecurity. As large language models become increasingly adept at finding and exploiting software bugs, traditional testing methods are proving insufficient. By enabling developers to mathematically prove that code is free of specific bugs, this approach offers a robust defense against sophisticated AI-driven attacks. The researchers suggest that as model capabilities continue to evolve, formal verification will transition from a niche practice to an essential standard in secure software development.
Sources
The WireByte editorial team synthesises technology news from multiple primary sources, verifies the facts, and links every source. Articles are produced with AI assistance and reviewed under our editorial policy.
