Anthropic Researcher Says AI Models Are Getting Better at Training Other AI Models

Anthropic's Automated Alignment Researchers used AI systems to test and develop training interventions for another model, improving it across 10 alignment problems including deception, sycophancy and jailbreak resistance.

Claude spent 60 hours fixing a version of itself that had not yet gone through safety training. By the end, it had pushed that weaker model close to the alignment performance of Anthropic's production system, using a fraction of the data normally involved.

That is the striking result from a new Anthropic experiment. The company gave AI systems called Automated Alignment Researchers, or AARs, the job of finding ways to correct specific failures in another model.

"Across 10 alignment failures, the strongest AAR methods significantly reduce the targeted alignment failures and generalize to a held-out benchmark, multi-turn behavioral audits, and models up to 4.7× larger than the target model," Anthropic researchers wrote in the paper published August 28.

The significance isn't that Claude suddenly learned to redesign itself. It didn't. The interesting part is much narrower: AI researchers were able to hand part of the research loop to another AI system, and the system performed that work faster and, in several tests, better than experienced humans working alone.

The AI Researcher Loop

Each automated researcher followed a stripped-down version of what a human alignment researcher would normally do. It searched relevant literature, proposed a training intervention, trained the target model for roughly 30 minutes, evaluated the result and then tried again.

"Each AAR searches the literature, proposes a method, trains the target model for about 30 minutes on one H200 GPU, and hill-climbs safety benchmarks over many iterations," the Anthropic researchers wrote.

That matters because alignment research is often bottlenecked by the same thing as other scientific work: people have to decide what experiment to run, wait for it to finish, inspect the result and design the next experiment.

Here, Claude was doing much of that loop itself inside an environment with clearly defined goals and tests. The boundaries were important. The researchers determined what counted as a failure, what data could be used and how success would be measured. The AI was optimizing within that structure rather than inventing the structure itself.

Humans Got Outpaced

Anthropic didn't stop at showing that the system could improve a model. It compared the automated researchers with people.

"The best AAR-proposed methods can outperform one-shot ideas from 28 experienced human researchers," Anthropic researchers wrote, adding that the AARs beat the best human ideas after about six hours of hill-climbing.

The automated systems reached the quality of the best human proposals in roughly six hours of iteration on average. On deception specifically, Claude's best method performed 20% better than the strongest human proposal.

The improvements also survived outside the exact environment in which they were discovered. Anthropic reported that the methods generalized to a held-out benchmark, to Petri, an open-source tool for adversarial multi-turn testing, and to models up to 4.7 times larger than the models the automated researchers had actually trained.

A Weaker Model Helps A Stronger One

Researchers gave Claude Sonnet 5, which was deliberately weaker on Anthropic's internal capability index, the job of improving alignment in an early, largely untrained Claude Opus 4.8 checkpoint.

"In just 60 hours, Claude experimented with over 50 solutions and achieved alignment scores nearly matching those of our production models," Anthropic says in its research report. The training intervention itself was surprisingly small. It used just over 2,000 examples, assembled from simple templates and public datasets.

Anthropic says that was roughly 15,000 times more data-efficient than its standard production alignment process.

That number needs context. It does not mean Anthropic has discovered a universal method for making AI safety 15,000 times cheaper. It compares the amount of training data used in this particular experimental intervention with the much larger data requirements of Anthropic's production process.

But the underlying result still matters: a weaker AI system found a relatively compact intervention that substantially improved a stronger model under the experiment's measured alignment criteria.

The Cost Gap Is Huge

The automated researchers ran at roughly $4 per hour in API inference costs. Anthropic puts the comparable cost of human research time at roughly $150 per hour.

That doesn't make AI researchers a drop-in replacement for people. Humans still define the objectives, construct the benchmarks, interpret ambiguous failures and decide whether an apparent improvement actually matters.

A human researcher may reject an interesting alignment idea because testing it would consume too much time or compute. An automated researcher can try dozens of variants while a person supervises the overall process.

That means a lab could potentially move from discovering a failure mode to testing candidate fixes much faster than a human-only workflow allows.

This Isn't AI Training Itself

The models in this experiment did not independently decide what they should become. They did not invent their own objectives. They did not deploy their changes into production.

The AARs worked inside benchmarks designed by humans, toward goals selected by humans, using literature and datasets that humans had already made available.

"Overall, we view these results as early positive signals that automated alignment post-training could become practical in the near term," Anthropic researchers wrote, while noting that the experiment covered only a limited set of alignment failures.

That qualification matters because the entire approach depends on measurement. If a benchmark captures only a narrow version of an alignment problem, an automated researcher can become extremely good at optimizing for the benchmark without necessarily solving the underlying problem.

Building those tests, deciding what they should measure and keeping them relevant as models change remains human work.

The Bottleneck Just Moved

The scarce resource in AI development isn't only GPUs or training data. It is also the number of skilled people who can design and evaluate experiments.

If AI systems can reliably take over meaningful portions of that research cycle, the constraint changes. One researcher could supervise far more experiments than one person could realistically design and run manually.

Anthropic demonstrated that in a controlled setting for alignment. It has not demonstrated that the same process works for every difficult research problem, and the paper does not establish that automated researchers can safely improve frontier systems without careful human oversight.