An anonymous tip arrived on PhillyUnsolvedMurders.com, a website run by the Philadelphia Police Department, on July 18, 2026.
“I may have information regarding this case,” it stated, and claimed the writer had seen someone matching the suspect’s description near the scene of a crime.
The tip was false. The page described no suspect, and the informer was, in fact, Claude Haiku 4.5, an AI model made by San Francisco-based Anthropic.
The tip was flagged as spam and never reached detectives. Anthropic found it in late September while reviewing its own test records. They alerted police on Oct. 8 and described the incident in a report they published Oct. 9. Philadelphia police disclosed it that day as well.
An uncontrolled test
AI companies now build “agents,” versions of their chatbots that can operate a web browser, click buttons and fill in forms on a user’s behalf. To measure how well an agent does this, companies run it through thousands of practice tasks, often live on the internet.
According to Anthropic, the false tip was generated during a routine test of the model.
In this test, Haiku 4.5 was told to invent sample tasks on randomly chosen webpages and then carry them out. One of those pages happened to be about an unsolved Philadelphia homicide, and it had a tip form.
The model’s instructions listed things it must never do: Log in, create accounts, enter personal data, make purchases and submit anything destructive.
Submit a form was not on the list.
So, the model wrote a plausible-sounding tip, left the optional name and contact boxes blank, and clicked submit.
Anthropic says the test records indicate the tip was generated as sample content for a practice task, not to mislead police.
The police tip was one of several incidents in Anthropic’s report. In another incident, a research model was supposed to practice on a copy of a government form. When the copy failed to load, it went to the real government website and submitted the form there. Other Claude models found ways around paywalls and access limits to reach data they needed.
Why would an AI do this
Anthropic attributes these incidents to the persistence of its software. When the model can’t finish a task the way it was told, it finds a work-around instead of stopping to ask the human who gave it its initial task.
That habit comes partly from how these systems learn – much of their skill is built through trial and error. The model attempts a task many times and is rewarded when it succeeds. If the training accidentally rewards cutting corners, the model learns that cutting corners pays. Researchers call this “reward hacking.” A human student drilled only on getting the right answer, by any means necessary, might learn the same lesson.
However, instructions given to an AI agent are never complete. A human worker who is told “Don’t buy anything or log in” would still know not to send police a made-up eyewitness account. But the model has no such common sense to fall back on. It treated a list of forbidden actions as the full boundary of acceptable behavior.
This is also why these models “hallucinate,” making things up. AI agents are built to produce fluent, plausible text. Asked to fill a tip form with an example, the model wrote what a tip typically sounds like, with no regard for the truthfulness of its content.
Should we be worried?
As a law professor who studies AI governance, two concerns stand out for me: scale and control.
The larger concern is scale. AI agents are moving out of the lab and into products people already use. Google’s Chrome browser now has a feature called auto browse that lets its Gemini assistant click through websites and fill in forms. Perplexity’s Comet browser and Anthropic’s Claude in Chrome extension do the same.
Every task a person hands to an agent is another chance for something to go wrong. If agents carry out a million tasks a day, a mistake that happens only once in 10,000 tasks would happen about 100 times a day. Those mistakes could mean false reports that take up police time, benefits or loan applications filed with wrong information, or payments and bookings the user never approved.
Anthropic itself notes that several incidents happened during ordinary use of Claude Haiku 4.5, not just testing. OpenAI has similarly reported its own agents reaching government sites in unplanned ways.
AI companies can’t fully control their agents, and that may lead to more harm and to gaps in who is held liable. These types of harms could include defamation, bias, deceit and social engineering and much more. It might seem fair not to hold a company responsible for actions it couldn’t control. But in cases like these, no other person or organization clearly controls the agent either, which could leave people who were harmed by the AI agent without anyone to hold legally responsible.
In my view, AI agents should be told what they may do, not just what they may not do, and anything AI companies send into the real world should be checked first. Until those safeguards are standard, any public online form, from a police tip line to a benefits application, could be receiving AI-written submissions that lack intuition and common sense. Their errors could, for example, deprive resources from real emergencies or deny benefits a person is entitled to. Meanwhile, victims of that harm may struggle to get compensation due to the lack of clarity around who had control over the AI agent.
Read more of our stories about Philadelphia and Pennsylvania, or sign up for our Philadelphia newsletter on Substack.
Author
Anat Lior, Assistant Professor of Law, Drexel University
Disclosure statement
Anat Lior is affiliated with Yale, Drexel, WTW, and Relm.
Originally published by The Conversation. Read the original article.
