Anthropic calls for a pause in global AI development

As leading AI systems grow more capable, Anthropic is urging the world to consider a coordinated pause on the development of the most advanced models, arguing that early warning signs suggest the technology could eventually slip beyond human control.

The San Francisco-based company behind the Claude family of AI models said in a new report that a global slowdown in frontier AI work would “likely be a good thing”. But it also stressed that a unilateral move would accomplish little, because competitors would almost certainly press ahead.

“We believe it would be good for the world to have the option to slow or temporarily pause frontier AI development to enable societal structures and alignment research to keep up with the advance of the technology,” it said.

For any genuine pause to hold, Anthropic said, several major AI developers across multiple countries — especially the US and China — would need to halt development simultaneously under rules that could be credibly monitored and enforced.

That position has drawn criticism from other corners of the AI industry and from officials in the White House, who argue Anthropic leans too heavily on extreme-risk scenarios and uses safety language to justify slowing its rivals.

Anthropic is led by former OpenAI employee Dario Amodei

Even so, the White House has recognized the strength of Anthropic’s Mythos model, which has not been released to the public because of its cybersecurity capabilities and is currently limited to a small group of vetted organizations.

The idea is likely to face stiff resistance in both Washington and Silicon Valley, where policymakers and technology leaders have repeatedly warned that easing off AI development could give China a critical strategic advantage in what many regard as the century’s defining technological contest.

US President Donald Trump, however, said he raised the prospect of working with China on AI safety during his recent trip to Beijing.

Mr Trump also signed an executive order this week giving the government 30 days to carry out a preliminary review of the most powerful US AI models before they are released.

‘Human role narrowing’, says Anthropic

Anthropic likened the challenge to nuclear arms control, while cautioning that AI would be even more difficult to regulate because training runs are much easier to conceal than missile silos, and the incentive to continue in secret would be immense.

The company said it intends in the coming months to convene government officials, scientists, advocacy organizations and rival AI companies to explore how such an arrangement might function in practice.

Its push for international coordination comes as the company says its internal data shows AI is already sharply accelerating the creation of more AI.

That dynamic, Anthropic warned, risks creating a feedback loop that could ultimately produce what researchers describe as “recursive self-improvement”.

In simple terms, that would mean an AI system becoming able to teach itself how to improve, with only limited human involvement.

“We are not there yet, and recursive self-improvement is not inevitable,” the report said, while warning that the moment could arrive sooner than most governments and institutions are prepared for.

“The evidence suggests that the human role is narrowing at each step in the AI development process,” the company said.