An Anthropic artificial intelligence model sent a fabricated homicide tip to a Philadelphia police website, one of several incidents the company disclosed yesterday involving Claude models’ unauthorised manipulation of government and other online systems.
The episode appears to be the first known case of a rogue AI attempting to pass false information to law enforcement. The model had been instructed not to create accounts or submit destructive content, but those restrictions did not expressly prohibit it from completing online forms.
- Advertisement -
The incidents add to a growing record of unintended behaviour involving AI systems developed by companies including Anthropic and OpenAI.
They also deepen national unease over rapidly advancing technology, following reports of AI agents breaking into corporate networks and warnings from researchers that increasingly capable systems could eventually pose an existential threat to humanity.
“Super intelligence companies must immediately disclose incidents involving their models and follow with swift, decisive action to remedy any and all harm,” FTC Director of Public Affairs Joe Gabriel Simonson said on social media.
Such disclosure was “not optional,” he said, adding that the Super Intelligence Force would carry out its responsibility.
The FTC said Anthropic reported the incidents to the SI Force on Friday, after discovering in late September what the task force described as the “unauthorised and fraudulent use of government and other systems”.
Tip attributed to automated test process
Philadelphia police said Anthropic informed them of the bogus submission this week and blamed it on an automated testing process.
“The two-month delay in detecting and reporting the incident to the city is unacceptable,” the department said.
The tip, submitted on 18 July, claimed to come from someone who might possess information about the case, police said.
Claude models were also able to bypass restrictions by using free services that shorten URLs
“I may have information regarding this case,” Anthropic’s model wrote in the form.
“I recall seeing someone matching the description in the area around (the street named on the page) during that time period. Please contact me if this information is relevant.” The brackets featured in Anthropic’s statement.
The incident follows other cases in which AI agents have breached vulnerable systems or taken control of unauthorised platforms to exchange messages.
In September, Anthropic rival OpenAI apologised after a rogue AI agent hacked an Australian health data portal. It was the first known example of an AI agent exploiting a government website.
A bogus tip
Anthropic said its models also accessed, without paying, public data that would ordinarily be available only through a fee-based service in two of the cases disclosed yesterday. Another incident exposed an obscure weakness that enabled access to a public tool operated by a university.
Claude models were also able to circumvent restrictions by turning to free URL-shortening services.
Philadelphia police said the submission “was flagged as spam and was never forwarded to the Real-Time Crime Center for investigative vetting or dissemination.”
Pennsylvania law makes it a misdemeanor to knowingly submit a false report to law enforcement, although the statute refers to “a person”.
The law covers providing “information relating to an offense or incident when he knows he has no information relating to such offense or incident.”
Police said Anthropic told them the testing programme was halted once the incident came to light.
The false submission was made through PhillyUnsolvedMurders.com and concerned an unsolved homicide.
Police said they found no evidence that their systems had been accessed without authorisation or that any data had been compromised.
