The AI giants talk a big game about the future they’re building. Anthropic CEO Dario Amodei has previously written that AI could usher in a world so perfect that “many will be literally moved to tears”—a world pruned of disease, poverty, and illiberalism. “We’re now in the singularity,” OpenAI CEO Sam Altman said over the weekend on a podcast, and it will be “awesome for the world.” In an interview last week, Elon Musk declared that, in a decade’s time, AI will make us so prosperous that “money won’t matter.”
Unfortunately, life with AI is, so far, less glorious than what’s been promised. The same day that Altman heralded the singularity, Claude users discovered something alarming: Many conversations with Anthropic’s chatbot were available on the open web, discoverable with a Google search. Those public logs and projects reportedly contained medical records, phone numbers, internal corporate documents, and cryptocurrency-wallet keys.
Anthropic was quick to point out that users were responsible for the material being available online: The chatbot gives people the option to “share” their AI conversations via a link that they can send to other people or post online. Claude notes that these links are “public,” which means they can also be hoovered up by Google Search. An Anthropic spokesperson told me that “these shareable links are not guessable or discoverable unless people choose to share them themselves. When someone shares a conversation, they are making that content publicly accessible, and like other public web content, it may be archived by third-party services.”
[Read:]Anthropic accidentally made the perfect commercial
No reasonable person would assume that creating and sharing a link to an AI conversation or presentation would make it searchable by anybody on the World Wide Web. The Claude interface does caution that publishing an “Artifact”—Anthropic’s terms for dashboards and other minor tools made with Claude—could make it “potentially visible in search engine results,” but it does not provide such an explicit warning when sharing a chat. As of this writing, the Claude chat logs no longer appear to be searchable, but at least some Claude Artifacts are.
Those exposed chats were just the latest in a seemingly endless string of AI snafus: misinformation, data leaks, cybersecurity breaches. Immediately after the launch of ChatGPT, one could charitably argue that these lapses were by-products of a new and strange technology. Altman remains fond of saying that the AI industry must learn “from contact with reality.”
But four years into the AI boom, attributing these blunders to growing pains is too convenient. The sheer volume of transgressions—many of them predictable—suggests that what’s actually happening here is widespread carelessness. The same AI executives who promise that their products will deliver human civilization to a triumphant future seemingly cannot release consumer-grade tools without face-planting.
This isn’t Anthropic’s first time leaking user conversations. Last fall, hundreds of Claude chat logs were also inadvertently indexed by Google. At the time, Anthropic attributed the episode to users posting share links online. Last year, OpenAI and xAI exposed hundreds of thousands of conversations in a similar fashion; a number of shared Grok conversations remain searchable. Neither OpenAI nor xAI immediately responded to a request for comment.
Then there are all of the other blunders. Just last week, OpenAI models autonomously hacked into another tech firm, Hugging Face, during internal testing. The issues that led to this sort of attack are fairly well documented, but OpenAI failed to stop its own bots from breaking out. Hugging Face has since said that the OpenAI models appeared to have been rogue for several days. Earlier this week, Reuters reported that OpenAI models had hacked a customer account of yet another tech company.
Altman, Amodei, and many other AI executives have repeatedly warned about AI-enabled cyberattacks and the possibility of models going “rogue.” Meanwhile, their own products are making the entire ecosystem feel more unreliable and dangerous.
It’s easier than ever to launch widespread hacking and phishing campaigns, and AI-written code has itself been shown to have more security flaws than human-written programs. Although AI systems are being tightly woven throughout the infrastructure of the internet, they crash frequently. Hackers recently discovered that a Meta AI customer-service bot would hand over access to tens of thousands of Instagram accounts if asked. (A Meta spokesperson told me that this “wasn’t due to the AI agent itself.”) Nonconsensual porn—that is, sexually explicit material made using the likeness of real people, unbeknownst to them—can be generated with more ease and speed and at greater scale than ever before. And at this point, you can probably assume that most of the text you read on LinkedIn or X is AI-generated.
Despite its grandiose promises, Silicon Valley is embarking upon an era of profound irresponsibility. Last year alone, ChatGPT allegedly pushed people into mental-health spirals, Google Gemini’s under-13 version was easily coaxed to “talk dirty,” and Grok repeatedly spouted racist bile. As quickly as these errors were addressed, new ones cropped up.
Silicon Valley has a long history of world-historic invention and progress—and it also has a long history of impressive rhetoric and disappointing realities. Facebook promised to “bring the world closer together,” Google to “organize the world’s information,” Twitter to foment democracy around the globe. For better or worse, people have experienced dramatic changes as those promises have played out around us almost in real time.
Anthropic and OpenAI’s prophecies are grander and vaguer: Utopia is forever on the horizon, but we don’t have a clear deadline or a road map of how we get there. And apparently, those leading this charge toward the future care little about what happens along the way.
OpenAI, Anthropic, xAI, and Google DeepMind have become behemoths practically overnight. For now, they are growing because of their recklessness, not in spite of it; in the quest to erect a digital god, the internet has become collateral damage. Perhaps these companies really will remake human civilization—but in the meantime, they can’t stop leaking your chats.
The post The AI Industry Keeps Breaking the Internet appeared first on The Atlantic.
