Anthropic CEO Dario Amodei Urges Slowing AI Development as Altman and Musk Agree

Anthropic CEO Dario Amodei on the 12th (local time) published an essay arguing that the pace of artificial intelligence (AI) development should be slowed to prepare for the risk of AI slipping out of human control. REUTERS/Dado Ruvic/Illustration/Provided by Yonhap News

As competition among artificial intelligence (AI) companies accelerates, top executives (CEOs) at leading AI firms are calling for slowing the pace of AI performance improvements and strengthening safety evaluations. They worry that as AI approaches the stage where it can improve its own capabilities, humans will no longer be able to understand and control the technology quickly enough.

According to major U.S. outlets including the New York Times and the Washington Post on the 12th (local time), Anthropic CEO Dario Amodei argued in an approximately 3,800-word essay that the pace of AI development should be slowed worldwide.

Amodei has been warning about the dangers of AI for years. His latest essay is being described as one of the most urgent and forceful calls for action ever issued by a major AI company CEO.

He wrote, "In the past few months, I've become convinced that investing in risk prevention alone is not enough to properly address AI risks, and that we must regulate the pace of AI capability advances," adding, "We have to slow the rate at which we improve AI models' capabilities." The goal, he said, is to buy enough time for safety measures to catch up with technological progress.

He continued, "Even if we slow down, it will still feel like we are progressing rapidly," and added, "We must use the time we gain wisely."

What AI company CEOs point to as a key risk factor is the possibility of "recursive self-improvement," in which AI systems advance themselves without researchers' help. Some AI developers have already warned that once AI repeatedly improves its own performance, it could move beyond human control.

Amodei stressed that, "If we leave the current situation as it is, it could outpace humans' ability to understand and control these systems," and said that we must proceed with extreme caution.

He proposed introducing a system of "internal evaluators." Instead of leaving AI safety entirely to companies' internal self-regulation, external experts should be granted access to the inside of companies to assess their safety levels, he argued.

He also argued that democratic nations should jointly establish AI safety standards that can, when necessary, be translated into regulation. For any effective adjustment of development speed, he said, global cooperation will be needed, including with authoritarian countries.

Sam Altman, CEO of OpenAI, and Elon Musk also agreed with Amodei's position. Just a few hours after Amodei posted his essay, Altman wrote on social media that it was "a topic we've been heavily discussing at OpenAI in recent weeks" and said that "slowing development is essential."

He said he supports the idea of having an independent evaluation body with access rights similar to those of employees, and stated that OpenAI would follow suit. Musk wrote on his social media platform X that "Dario is right."

Within Silicon Valley, some critics argue that Amodei's remarks are strategic. They claim that AI companies emphasize safety issues as a way to strengthen their market dominance.

Some point out that if large companies like Anthropic and OpenAI highlight AI risks and demand regulation, small and medium-sized firms with weaker regulatory compliance capabilities may find it difficult to enter the market.

NVIDIA CEO Jensen Huang, at the Goldman Sachs technology conference in San Francisco on the 10th, criticized AI companies for deliberately amplifying security concerns in order to launch new cybersecurity products.

While emphasizing that "the probability that AI will bring enormous benefits to humanity, such as treating diseases and boosting economic growth, remains high," Amodei also said, "Even so, the risks of AI are too great." He added, "It will not be easy to implement the measures I've proposed to develop AI at a safe pace, but I believe we have an obligation to try for the sake of humanity."