Anthropic proposes blocking AI models if more than 50 people risk dying — TechCrunch

In the United States, Anthropic is circulating a draft artificial intelligence safety bill that provides for the possibility of blocking the release of an AI model through a court. The basis for such a step would be a foreseeable and imminent catastrophic risk under which more than 50 people could die or property damage exceeding $1 billion could be caused, TechCrunch reports.

What risks the draft covers

According to the publication, this concerns dangers associated with the creation or release of biological weapons, a model going out of its developer's control, or a cyberattack. The criteria of more than 50 potential deaths and $1 billion in damages are similar to those used in California's enacted Senate Bill 53, where such scenarios are defined as catastrophic risk.

It is proposed that a model's potential to cause a catastrophe be assessed by independent evaluators registered with the U.S. Department of Commerce. The AI laboratories themselves may pay them, provided that the compensation does not depend on the assessment results.

More current news is available on the UA.News Telegram channel Telegram.

Decisions only through a court

If evaluators determine that there is an imminent catastrophic risk, they will have 72 hours to notify the Department of Commerce. The agency would then have to ask a judge in the District of Columbia to halt the model's release.

The draft explicitly prohibits federal agencies from independently deciding to stop the distribution of an AI model. An Anthropic representative confirmed the existence of the document to Politico and said it reflects proposals from the Advanced AI Framework, which the company has promoted among federal and state lawmakers since June.