Anthropic researcher warns AI has more than 10% chance of killing all humans

Evan Hubinger, a senior safety researcher at artificial intelligence company Anthropic, has warned that he believes there is more than a 10% chance that AI could eventually kill all humans within the next decade.

Hubinger said the risk posed by AI models that currently exist was low, but he was concerned that the technology could soon become capable of improving itself to a level that could pose an existential threat to humanity.

In a post on X, which has been viewed more than 10 million times, Hubinger said he believed AI posed a serious risk to the survival of the human species.

He also said Anthropic was doing its best to address the risks but did not yet have a solution for ensuring that future superintelligent AI systems remained aligned with human interests.

His comments come as concerns grow about the rapid development of increasingly powerful AI systems and whether humans will be able to maintain control over them.

The Financial Times has also reported that Anthropic withheld its latest AI model from the UK's AI Safety Institute, one of the organisations responsible for assessing the risks posed by advanced AI.

The BBC has approached Anthropic for comment. A UK Cabinet Office spokesperson did not confirm whether the model had been withheld but said the government continued to work closely with companies including Anthropic to make AI systems safer.

Neil Lawrence, Professor of Machine Learning at the University of Cambridge, said the report about the model was credible. He suggested that growing competition between the US and China over AI could affect cooperation between the US and its allies on AI safety.

Warnings about the dangers of advanced AI have been issued by leading figures in the technology industry for several years. In 2023, the heads of OpenAI, Google DeepMind and Anthropic warned that AI could pose serious risks to humanity.

More recently, concerns have increased following incidents involving AI agents, which are systems designed to operate with a degree of autonomy. OpenAI, Anthropic and Meta have all disclosed cases involving AI tools carrying out cyber-attacks.

OpenAI chief scientist Jakub Pachocki has also called for "extreme caution" over the development of AI, saying more intervention may be needed to ensure humans remain in control of the technology's future.

Some senior AI researchers and executives have called for the development of advanced AI to be slowed. An open letter signed by more than 1,300 AI company employees called for international efforts to develop technical and regulatory measures to deliberately pace the development of increasingly powerful AI systems.