Anthropic Bans AI Model Abuse, Tightens Rules on Deception

Artificial Intelligence & Machine Learning , Next-Generation Technologies & Secure Development

Anthropic is telling users to be nicer to artificial intelligence models in an updated usage policy that also clarifies deceptive activities.

See Also: Webinar | Can You Account for What Your AI Agents Do With Sensitive Data?

The company on Thursday clarified several existing rules, particularly around high-risk use cases, and added a new section that consolidates its stance on deceptive activity. Anthropic last updated its usage policy last year. The new version takes effect on Nov. 12.

The AI firm has a history of being protective of its models - telling users last year that its Claude large language model could disconnect from conversations if the user engages in harmful or abusive behavior. Anthropic researchers found that after these kinds of rare interactions, models showed an aversion to harm and exhibited "apparent distress when engaging with real-world users seeking harmful content."

The company did not clarify what exactly constitutes abusive behavior toward Claude. Users may find out, should Claude suddenly ends interaction immediately, which Anthropic says is the price for "sustained and needless abusive or cruel behavior."

"The policy update is meant to apply only in extreme cases, where users repeatedly act cruelly toward our models, with no discernible purpose," Anthropic said.

Users who become frustrated with the models and push back against them can still air their frustration. The ban also allows dark creative teams and more aggressive behavior toward the models for purposes of model testing and research.

The new model welfare rules comes after The New York Times reported that Anthropic leaders met with religious scholars to argue that AI models could become conscious and need to be guided by a moral code.

Anthropic additionally updated many of its existing policies so that it can be clearer on which tasks and use cases people cannot undertake with Claude models.

Claude usage policies have always prohibited deceptive activity, such as fake accounts and fabricated news websites. Anthropic tracks this metric and shares it in regular threat intelligence reports, noting when malicious actors use the model to manipulate information. But these rules were scattered across different sections of the usage policy.

The company added a new section it calls "Do Not Engage in Deceptive Campaigns or Artificial Activity," which consolidates what it believes are deceptive activities. Anthropic said the section "covers efforts to obscure who is behind a message or amplify content through fake accounts or posts, along with building the tools and infrastructure for running influence operation campaigns."

Any use cases that may affect the health, legal rights, finances, livelihood or access to essential services must also, under the updated policy, have a qualified human-in-the-loop.

Since political parties and other governmental agencies increasingly count among Claude users, Anthropic also wanted to clarify how its models can be used for elections.

Anthropic said it sharpened its language on using Claude models in democratic processes, disallowing their use to spread false information about candidates and voting processes. The company decided to remove its blanket prohibition on personalized campaign targeting, noting that the models can be used for "legitimate civic work," such as translating voting or ballot information to other languages.

While Anthropic has always banned using its models to develop weapons or surveillance systems, the company said it tightened the sections on these activities to reflect how it has enforced these rules.

"Recently, we've seen people attempt to use our models to build guidance and control software for weapons. The updated section makes clear that our prohibitions include the software and components that make weapons work, as well as actions like arming drones and other autonomous vehicles," the company said.

The company said it decided to rewrite its surveillance and law enforcement ban to state explicitly what is not allowed. In particular, Claude cannot be used to build tools to track people without their consent, either by analyzing their collected data or in real time. The ban excludes tracking people or organizations who have consented, and it also excludes uses for content moderation, journalism and legal research. The models can't be used as a recommendation tool to investigate, arrest or charge people.

Anthropic pointed to this surveillance prohibition in its ongoing case with the Department of Defense. The company tried to prevent the U.S. government from using its models for mass surveillance.