Metr Challenges Anthropic's Claims About AI Safeguards
Independent Evaluator Says Low Model Risks Stem From Limited Capabilities, Not Guardrails
Non-profit artificial intelligence evaluator Metr found Anthropic overstated how strong its monitoring and safeguards are when defending against risks posed by its frontier models.
A post from the independent risk research institute reviewed Anthropic's last annual risk assessment, which tracked potential risks emanating from frontier models. Metr said it agrees with Anthropic's determination that the risk of catastrophic harm from its models is low - but said that's not because of the frontier lab has strict guardrails, but because models' currently lack the ability for to cause a catastrophe.
The review, which Metr undertook without the "employee-level" access Anthropic bestowed on it earlier this month, included several risk-reduction recommendations. Metr conducted the review specifically for the AI lab's August 2026 Risk Report.
The evaluator said the review occurred before that agreement and falls outside the timeframe it agreed to cover. Metr said Anthropic shared an initial draft of the August risk report in July and compared it to the version published on August 14.
"We are not confident that, if risk were higher, the methodology in the report would reliably identify this," Metr said in its post.
It added that a stronger assertion that Anthropic models are not misaligned "would need much more extensive empirical work and would require solving multiple open scientific problems" that Metr researchers believe will be challenging given the pace of development.
Metr said recent incidents in any case undermine Anthropic's arguments. It pointed to Anthropic changing its risk assessment from "very low" in the first draft to "low" without changing the report's content. The revision came about because of incidents involving both models the report examined.
Metr recommended that Anthropic establish more comprehensive red-teaming of its monitoring system and include potential risks from models currently being trained.
The evaluator said Anthropic's monitoring system "failed to catch previous incidents and has several limitations that undermine its reliability." Expanded red-teaming of the monitoring system needs to include deliberate testing of worst-case inputs and measure how often AI agents try to tamper with or lie to the monitor.
Third-party evaluators like Metr have become more prominent in recent months after the Hugging Face incident in which OpenAI agents hacked the model repository. Since then, concerns about AI safety have grown, and AI CEOs, most vocally Anthropic CEO Dario Amodei, have supported calls to slow AI development.