Anthropic's 'Mythos' AI proves that obsessing over AGI is folly

For years, progress in AI has been motivated by an industry-wide yen to create software that's at least as capable as humans -- not at some tasks, but all of them. The precise definition of the goal varies, and two maddeningly overlapping terms, artificial general intelligence (AGI) and superintelligence, both get bandied around. But no matter how you look at the aspiration (or how long you think it will take to achieve), it's about the ways the world will change when software can do everything extraordinarily well.

I've written -- here and here -- about why I believe fixating on that eventuality isn't the best way to think about AI and its impact. It might turn out that AI trounces humanity at some jobs and never rivals it at others. That would not be reason to take it any less seriously. This week brought some of the clearest evidence of that point so far.

On April 7, Anthropic announced a new version of its Claude model called Claude Mythos Preview. Like existing Claude versions such as Sonnet and Opus, it was trained for general competency, not to be a specialist at anything in particular. But Anthropic says that when it tested Mythos, it discovered it had made dramatic strides in coding ability. It was particularly good at finding and exploiting vulnerabilities in existing software, surpassing "all but the most skilled humans."