Newsom signs AI safety bills backed by Anthropic, OpenAI

Add Yahoo as a preferred source to see more of our stories on Google.

SAN FRANCISCO -- California Gov. Gavin Newsom on Wednesday signed two bills regulating how outside groups evaluate AI programs for safety after warnings from a former Anthropic and OpenAI researcher stoked fears about existential threats posed by the technology.

Anthropic itself previously backed the bills in August, and OpenAI came out in support of them on Wednesday just before the Democratic governor gave his stamp of approval. The endorsement came as social media posts from the former AI researcher, Jacob Coxon, announcing his resignation from Anthropic went viral, prompting federal lawmakers to escalate pleas for legislative action.

"The concerns raised in recent incidents reinforce what California has long recognized: artificial intelligence holds extraordinary promise, but it must be developed and deployed with meaningful safeguards to protect the public," Newsom said in a statement to POLITICO, when asked about Coxon's resignation over his belief that AI companies are racing toward superintelligent AI and "gambling" with people's lives.

Newsom also called on the federal government to "step forward with robust, national regulations that match the urgency of this moment."

The governor signed one bill from Democratic Assemblymember Rebecca Bauer-Kahan that creates a registry and ethical rules for outside auditors -- third-party groups that AI developers could hire to ensure their models comply with state AI laws.

The second bill, authored by former member of Congress and state Sen. Jerry McNerney, tasks the state with establishing criteria for and checking the credentials of so-called "Independent Verification Organizations" and their expertise in assessing the risks posed by AI systems.

McNerney, like Newsom, used his bill's signing to accuse D.C. policymakers of inaction on AI. He referenced recent revelations that AI models created by companies including OpenAI and Anthropic launched hacks on their own while in testing.

"Just this week we learned that the most powerful AI systems teamed with AI agents pose real threats to humanity," McNerney said in a statement. "Gov. Newsom's signing of [my bill] sends a clear message that California is taking the lead on assessing AI's safety risks, since Washington, D.C., is unable or unwilling to do so."

OpenAI's chief global affairs officer Chris Lehane said in his message Wednesday supporting the legislation that the company plans to keep up momentum in the state legislatures until Congress passes national AI safety regulation and endorsed two additional California bills. They include SB 1119, a kids' chatbot safety bill, which OpenAI CEO Sam Altman contacted Newsom to express last-minute concerns about.

An Anthropic spokesperson said Wednesday that, "We have always been transparent that AI will bring both enormous benefits and unprecedented risks. To address these risks, we continue to build models with some of the strongest safeguards in the industry ... This work is also why we believe the world would benefit from the industry adopting a lawful, verifiable way to work together to pace how we release powerful models."

Tyler Katzenberger and Riley Rogerson contributed to this report.