With mounting pressure on AI labs to curb the many real-world harms their products cause consumers, it’s never been clearer that they’re monitoring users’ conversations.
Just ask Carli Michelle Heller, a woman arrested in Bonita Springs, Florida after allegedly telling Anthropic’s chatbot Claude about her plan to “shoot up” a nearby sheriff’s office.
According to WINK News, Heller was charged with making a written threat of violence — a second-degree felony in Florida — after Anthropic’s internal monitoring software flagged her chat logs for human review. After being viewed by a human employee at Anthropic, WINK reports, the company determined that Heller’s plans rose to the level of a credible threat, ultimately informing local police.
As Lee County Sheriff Carmine Marceno told local reporters in a statement, the woman admitted to using Claude like a “diary,” apparently unaware that her chat history could be read by Anthropic employees.
There’s a genuine tension between user privacy and the responsibility of AI companies to try to head off violent events. In the aftermath of the February mass shooting in Tumbler Ridge, Canada, for instance, it quickly emerged that the perpetrator had relied heavily on ChatGPT to plan her shooting.
Not only had OpenAI staffers known about those chat logs prior to the shooting — they had been flagged for human review months prior — but company executives even tried to hide and later downplay their product’s role in encouraging the deadly attack.
While Anthropic’s own track record on real-world harm is wretched, it seems the company is at least trying to avoid the same kind of incident — though with near-endless chatbot options now on the market, it could already be too late to prevent the next AI-assisted tragedy.
More on Anthropic: Cult of Anthropic Insiders Rumored to Be Worshipping Claude as a God
The post Anthropic Reports User to the Police appeared first on Futurism.
