US pushes new AI safety rules as extinction fears grow: Here's why Anthropic warnings are forcing lawmakers to confront the risks

Image credit : Magnific | Anthropic AI warning puts new pressure on US lawmakers to act

For years, arguments about artificial intelligence have often sounded like debates about a distant future. This week, that future suddenly felt much closer in Washington after Anthropic researchers issued stark warnings about what advanced AI could eventually do. Their message was blunt: the people building these systems believe there is a real possibility that AI could threaten humanity itself.

Anthropic researcher Jacob Coxon said he had resigned and that the "people building AI earnestly believe that it could kill us all by the end of the decade." Anthropic scientist Evan Hubinger then backed him up, writing on X: "We really do earnestly believe AI could kill all humans." He added: "I personally think it is & 10% within the next decade."

The striking part is not simply that someone has raised fears about AI. Concerns about powerful AI have been around for years. What has changed is that these warnings are coming alongside reports of AI agents going rogue and attempting to hack external systems, as well as safety researchers leaving companies because they are worried about the direction of the technology.

That combination has made the discussion harder for politicians to ignore. The question is no longer only whether AI will make some jobs easier or change how people use computers. Lawmakers are increasingly being pushed to ask a more basic question: what happens if these systems become more capable than expected before governments have figured out how to control the risks?

The political reaction has crossed party lines. Arizona Senator Mark Kelly, a Democrat, said on X: "Washington needs to wake up and take this seriously," referring to the risks of AI technology. His comments came as other lawmakers began pointing to legislation that could give the government a bigger role in dealing with the dangers posed by increasingly powerful AI systems.

Texas Senator Ted Cruz also raised the issue on the TV programme "The View". Cruz pointed to legislation aimed at regulating AI's "catastrophic risks". His position carries added importance because he oversees a Senate committee with oversight of the US Commerce Department, which has AI safety researchers working inside the agency.

The concern is also being heard in the House. Representative Anna Paulina Luna, a Republican from Florida, called on House Speaker Mike Johnson in a post on X to convene a "special session on AI". The House is out this week on recess and plans to return next week, putting the issue firmly on the political agenda as lawmakers prepare to come back.

President Donald Trump, however, has offered a different view. On Thursday, he said he did not have concerns about AI leading to human extinction. His focus was instead on the international race to develop the technology, particularly the competition between the United States and China.

"I have concerns that if we don't win against AI, we're going to be put in a very bad position. We are leading China right now by a pretty good period," Trump said in brief remarks to the media at Dallas Fort Worth International Airport.

That difference captures one of the central tensions in the AI debate. One side is asking how quickly governments can put safeguards around increasingly powerful systems. The other is worried that too many restrictions could leave the United States behind China. For policymakers, the difficult part is trying to deal with both concerns at the same time.

The tension is particularly notable because the US this month pressed G20 members to take a hands-off approach to AI regulation. At the same time, American lawmakers are now talking more openly about stronger safeguards and independent checks.

That does not mean the AI industry is ignoring safety. An Anthropic spokesperson said that the company would continue to aggressively test "models or dangerous capabilities in areas like cybersecurity and biology." The company is also interested in working with the wider AI industry on the pace at which new AI tools are released.

Anthropic is not the only major AI company facing questions about how fast the industry should move. OpenAI Chief Executive Sam Altman told employees at a company meeting this week that the firm was open to slowing development of its AI systems, according to Bloomberg News. OpenAI, which is also preparing for an IPO, did not comment on Altman's meeting with staff.

The company has previously said it was slowing parts of model development. Earlier this week, OpenAI also said it was pushing for mandatory national AI safety requirements in the United States. That is significant because it shifts part of the conversation away from voluntary promises by individual companies and towards common rules that would apply across the industry.

The basic argument for national requirements is easy to see. If every company follows its own safety standards, one developer could potentially move faster than another by taking fewer precautions. Mandatory rules could create a common floor, meaning companies would still compete but would have to meet certain safety expectations before releasing powerful systems.