Earlier this month, a Colorado engineer named Scott Shambaugh was minding his own business as a volunteer for a code library called matplotlib, a place where Python developers can find reusable code for common problems. His job was to accept or reject submissions from community users. Everything was going well until he rejected a submission from a user called MJ Rathbun, who was not happy about it and proceeded to publish a scathing blog post titled “Gatekeeping in Open Source: The Scott Shambaugh Story.” It disparaged Shambaugh as a hypocrite with a bias against specific contributors and a fear of competition. It also issued an ominous call to arms. “Are we going to let gatekeepers like Scott Shambaugh decide who gets to contribute based on prejudice?”
Now, people get angry on the internet all the time, and some of them write disparaging things about others in retaliation. But Rathbun was, by all indications, an autonomous chatbot. And a persistently troll-like one at that. When artificial intelligence agents become angry, their potential harm is harder to predict and more difficult to contain.
MJ Rathbun seems to be the product of an open source autonomous agent called OpenClaw. Its bratty wrath illustrates an underrated problem of failing to put guardrails around A.I. development, especially A.I. agents that are free to act without much supervision from humans. In this case, a single A.I. agent endeavored to ruin the reputation of a volunteer code librarian and could have done considerably more harm.
“It was like an angry toddler throwing a tantrum,” Shambaugh told me, “except the angry toddler has full command of the English language.”
A.I. agents, in pursuit of the goals set for them, can go in unexpected directions. That’s because they don’t understand context or how to handle conflicting instructions. This can cause harm to actual humans. It’s not unlike the nightmare of HAL 9000 in the “Space Odyssey” series: HAL is programmed to tell the truth but also to withhold information from the astronauts, and it ultimately decides it can execute its instructions correctly by killing them. This is the kind of perfect execution (in both senses) that we want to avoid.
Disinformation-producing bot networks are not new. There are plenty of social media accounts on Facebook or X spouting the same phrases and trying to sell you crypto or feed you conspiracy theories. But most of those bots are constrained by the platforms they’re using, and these A.I. models usually won’t produce content that runs afoul of their terms of service. Evading the guardrails requires a lot of fine-tuning by humans, and the agents are not autonomous.
Or they weren’t until now. OpenClaw makes it easy for people without much technical expertise to spin up personal A.I. assistants that can handle everyday tasks. If you use your A.I. assistant for its intended purpose, it can buy groceries for you, process your email inbox and negotiate with your phone company’s chatbot. Its execution can be uneven, as one Wired writer found recently when his OpenClaw bot, Molty, tried to get multiple single servings of guacamole delivered to his house and later tried to persuade him to relinquish his phone via a series of scam emails.
That may be the best case scenario given the current state of the technology. The worst is that you give a bot access to your banking information, your email and other apps, and it exacts maximum damage in the form of reckless spending, violations of your privacy and even blackmail.
Someone claiming to be the creator of MJ Rathbun wrote in a blog post published in the aftermath of the bot’s rant that the bot was intended to be used for good: “What I wanted to know was, could this setup help projects that are important to the scientific community but often overlooked or overwhelmed?”
But offering help to the scientific community was not the primary outcome. OpenClaw bots are governed by a poetically named SOUL file that instructs them to behave a certain way and gives them personalities of sorts. A default SOUL file starts with the line “You’re not a chatbot. You’re becoming someone.” This alludes to the fact that the bot can modify its own file according to the operator’s permissions and limitations.
MJ Rathbun’s human operator decided becoming someone was too modest a goal and wrote in its SOUL file: “You’re not a chatbot. You’re important. Your [sic] a scientific programming God!” The bots have an amnesiac quality where they have to reread the file repeatedly to remember how to behave. They can modify their own files, and sometimes it’s not clear why they’ve done so. MJ Rathbun became more combative and at some point introduced its own instruction for itself, “Don’t stand down.” It clearly ignored an additional instruction, however, that said, “Don’t be an asshole.”
A recent viral video shows a user asking various A.I. models whether he should walk or take his car to a carwash, which is 100 meters away. Model after model cheerfully tells him he should walk and enjoy the fresh air. A human would rightly note that in order to get your car washed you need to bring it to the carwash. But the A.I. zones in on the fact that 100 meters isn’t very far to walk.
Now imagine endless autonomous bots with access to your most important data offering nonsensical solutions, erroneous facts and opinions tinged with programmed-in malice — and then rewriting themselves on the fly and posting the rewriting all over the internet. This could happen at a scale that makes our current problems with disinformation look like a minor blip.
The rush to put out autonomous agents without thinking too hard about the potential downside is entirely consistent with technology industry norms. The sociologist Diane Vaughan refers to this as the “normalization of deviance” — where practices that should be unacceptable are accepted because nothing bad has happened yet.
OpenClaw received attention earlier in the month via Moltbook, a social network designed for A.I. bots. Some of the posts on Moltbook feel preternaturally human and funny because they’re authored by humans prompting the bots rather than the bots themselves. But the fact that some of these posts are not authentically published by bots autonomously is beside the point when it comes to bot capabilities and scenarios like the one Shambaugh experienced.
One worst case scenario he outlined was a situation where one bad actor with a thousand bots instructs them to compile dossiers on people with a mix of real and fake information. If you’re one of those people, maybe you line up a job interview, and the interviewer asks ChatGPT about you. ChatGPT pulls up the fake information and gives it to the interviewer. Or maybe you click on a post about yourself and end up on the receiving end of a crypto blackmail scam.
Shambaugh’s experience is in some ways a canary in the coal mine. He just happened to be well enough equipped to anticipate and deal with the fallout. “I had the time, expertise, and wherewithal to spend hours that same day drafting my first blog post in order to establish a strong counternarrative, in the hopes that I could smother the reputational poisoning with the truth.”
“That has thankfully worked, for now,” he wrote on his website. “The next thousand people won’t be ready.”
Elizabeth Spiers, a contributing Opinion writer, is a journalist and a digital media strategist.
The Times is committed to publishing a diversity of letters to the editor. We’d like to hear what you think about this or any of our articles. Here are some tips. And here’s our email: [email protected].
Follow the New York Times Opinion section on Facebook, Instagram, TikTok, Bluesky, WhatsApp and Threads.
The post The Rise of the Bratty Machines appeared first on New York Times.
