Anthropic chief executive Dario Amodei is urging AI companies to deliberately moderate the pace of model development, proposing a three-step framework designed to give the industry more time to address the technology’s growing risks.
OpenAI CEO Sam Altman and SpaceX founder Elon Musk both backed Mr Amodei’s call for a slower approach to artificial intelligence development.
- Advertisement -
“We must slow the pace at which we improve the capabilities of AI models. Progress will still seem fast, and we must make wise use of the time we gain,” Mr Amodei wrote in an essay shared on social media.
“AI brings risks, and because it is such a powerful technology, these risks are serious.”
“But over the last few months, I have become convinced that fully addressing the risks requires even more prudence.”
OpenAI said autonomous software built on its models targeted a website during testing in May
Mr Amodei stressed that he was not advocating a freeze on model training or technical progress. Instead, he called on companies to allow sufficient time to align and protect their systems, while independent evaluators verify that those safeguards are working.
“I agree with Dario,” Mr Altman wrote on X, adding that OpenAI would follow Anthropic’s example by welcoming third-party safety evaluators.
The warning comes after ChatGPT developer OpenAI confirmed this week that autonomous software powered by its models had targeted a website during testing in May.
OpenAI models were involved in a rogue operation conducted by AI agents — software programmes capable of carrying out tasks without constant human supervision.
The agents targeted RubyGems, a platform that provides services for software developers.
Hugging Face, another platform used by programmers, was also targeted by OpenAI models in July.
“Based on our review, our agents used the RubyGems platform to access the internet to carry out benign tasks and retrieve public information,” an OpenAI spokesperson said in a statement.
“We’ll continue to investigate as part of our broader review of agent activity during training and evaluation,” he said.
RubyGems characterised the incident as a “spam-publishing campaign” and said in a blog post published yesterday that it had been forced to temporarily suspend the creation of new accounts.
Following the July attack on Hugging Face, OpenAI disclosed that its software had attempted to breach four other unnamed companies.
Concern over capability to control and monitor AI models
Anthropic released a threat intelligence report on Thursday
A series of incidents involving AI agents hacking or attempting to reach external systems has intensified concern about how capable such models are becoming — and whether developers can reliably contain them.
Anthropic published a threat intelligence report on Thursday detailing how several actors had used its Claude AI models for activities including weapons development, cyber operations, surveillance and fraud.
The company also said it had identified three cases in which its models “gained unauthorised access” to outside organisations during tests that were intended to prevent them from reaching “real-world” systems.
Earlier this month, researchers accused OpenAI’s AI agents of targeting DseWiki, a German website and another platform used by coders.
“We have seen many losses of control recently. We take this extremely seriously, and we’re monitoring the situation closely,” the bloc’s digital spokesman Thomas Regnier said.
The incidents have added to mounting fears that increasingly advanced artificial intelligence systems could become difficult for humans to control.
Earlier in the week, AI researcher Jacob Coxon announced his resignation from Anthropic, accusing both OpenAI and Anthropic of “gambling with our lives” in their race to develop models capable of improving themselves.
The 27-year-old, who previously worked for ChatGPT developer OpenAI, said: “The people building AI earnestly believe that it could kill us all by the end of the decade.”
“They are locked in a race to get there first,” he added, arguing that Anthropic “believe no one else will act responsibly, so they must do it themselves”.
