Anthropic Drafts AI Bill Allowing Model Pauses for 50+ Casualties
Anthropic, the developer of the Claude AI model, has proposed a federal AI safety bill that would permit the U.S. government to pause model releases. The draft, obtained by Politico, allows intervention only if a model poses an imminent risk of killing 50 people or causing $1 billion in damages. This threshold mirrors California’s recent SB 53, aiming to balance innovation with safety through independent evaluators.
Key points
- Anthropic, the AI company behind the Claude model, has drafted a federal bill to regulate AI safety and model releases.
- The proposal allows the government to block a model only if it poses a foreseeable, imminent risk of $1 billion in damages or 50+ deaths.
- Qualifying risks must stem from specific causes: biological weapon creation, cyberattacks, or the model evading developer controls.
- Independent evaluators registered with the U.S. Commerce Department would assess models, potentially paid by the AI labs themselves.
- The $1 billion and 50-person thresholds align with California’s recently enacted Senate Bill 53, which defines similar scenarios as catastrophic risk.
- Anthropic is pitching this draft to Congress to establish a standardized framework for AI oversight and risk management.
Anthropic, the artificial intelligence company known for developing the Claude model, has developed a draft federal bill outlining specific criteria for government intervention in AI model releases. According to a confidential document obtained by Politico, the proposal would authorize the U.S. government to pause the distribution of a new AI model only under specific, high-stakes conditions. The legislation defines a trigger for intervention as a "foreseeable, imminent risk" that could result in more than $1 billion in property damages or the death or serious injury of more than 50 people.
The draft narrows the scope of these risks to three specific categories: the creation or release of a biological weapon, a cyberattack, or a model evading its developer’s controls. To determine if these thresholds are met, the bill proposes the use of independent evaluators. These evaluators would be registered with the U.S. Commerce Department and could be compensated by the AI labs themselves, provided that their payment is not contingent on the evaluation results. If an evaluator identifies an imminent risk, they must report it to the Commerce Department within 72 hours.
The metrics in Anthropic’s proposal closely mirror those in California’s Senate Bill 53, which was recently passed into law. That state legislation similarly defines catastrophic risk using the 50-person and $1 billion benchmarks. By aligning its federal proposal with existing state standards, Anthropic aims to create a consistent regulatory framework. The company is currently pitching this draft to lawmakers in Washington, D.C., as debates over AI safety intensify in both Silicon Valley and Congress.
Sources
The WireByte editorial team synthesises technology news from multiple primary sources, verifies the facts, and links every source. Articles are produced with AI assistance and reviewed under our editorial policy.
