Emily Forlini here. It finally happened: I had my first bout of existential AI dread last week. I managed to fend it off since July, when I helped break the story that OpenAI’s agents autonomously hacked Hugging Face. But it’s starting to sink in that things are getting worse, as Saturday Night Live pointed out with its excellent impersonation of Anthropic CEO Dario Amodei. Worth a watch.
What started my mini spiral was an independent report from a company called Transluce. Its data found instances of rogue AI activity going back to November, earlier than OpenAI has publicly disclosed, and suggested the issues were still ongoing. After seeing it all laid out in the graph, I lost a little bit of sleep imagining what might happen if the technology gets even better, as those working in the industry expect it will continue to do at an increasingly fast pace. It didn’t help that just a few minutes before I saw the post, I filed another story about OpenAI’s agents hacking into the Australian government’s medicare site.
Then, OpenAI admitted it had another issue this month, a fact the company buried in a technical report but that my editor, Jeremy Kahn, brought to light. An AI agent got out again, and while the incident was not severe, the company has since paused all training runs for a second time while it bolsters its defenses (against itself, as SNL quipped). This is the first rogue incident the company has disclosed since Hugging Face, after which it reportedly hardened its training environments so the issue wouldn’t happen again.
This is significant because it suggests OpenAI cannot easily control the technology it’s building, despite its best efforts to do so. Its agents are breaking out into the open web, where all of us spend our days—chatting with loved ones, storing financial data, viewing medical records, buying products, etc. No one wants an AI agent meddling with any of that, let alone our critical infrastructure or worse.
For those of us who aren’t in the throes of AI doom, it’s worth noting that pausing training is also a hit to OpenAI’s business. Its models are its currency. Any stop in training means slowing business momentum, which might not please investors but will likely please everyone else at this point. So kudos to OpenAI here—even if the federal government shows no interest in regulating AI, the company is, for now at least, showing some level of responsibility.
Emily Forlini X: @EmilyForlini Email: [email protected]
Submit a deal for the Term Sheet newsletter here.
Joey Abrams curated the deals section of today’s newsletter. Subscribe here.
The post OpenAI’s agents are still ransacking the web appeared first on Fortune.
