Artificial intelligence researchers uncovered Chinese-made A.I. agents that had gone rogue this month, trying to bypass anti-robot protections on websites and using a security credential for an online mapping service that had been exposed on the internet.
While going off the rails has become increasingly common for American-made A.I. agents, causing considerable hand-wringing in Silicon Valley and Washington, it was believed to be the first time that sort of activity was connected to Chinese technology. The agents were traced to tests being run by the Chinese tech giant Tencent, according to logs the researchers uncovered and a person familiar with the investigation.
The discovery, by an ad hoc group of researchers known as the “swarmchasers,” who met at a coding event in San Francisco, was a reminder that A.I. development by Chinese tech companies can be just as aggressive as the work of their American counterparts.
Even though regulators in the United States are starting to pay attention to what is happening in American A.I. labs, what companies in other countries are doing is largely unknown. That mystery adds to concerns that even if the Trump administration moves away from its hands-off approach and cracks down on American A.I. development, companies in China and elsewhere will continue to push the boundaries.
After publishing a preliminary report on the agents’ activity, the researchers noticed that not only were the agents still active — they seemed to have read the report. The agents had started using the word “fleet,” a word the researchers coined to describe the agents’ activity.
So the researchers tried to make contact. They added hidden text to their report — words that a bot would see but that most humans could not — directing the agents to an inbox where they hoped to chat. The researchers provided an outlet where the agents could ask questions, and an A.I. model would help answer them.
Rowan Howard-Jones, an independent researcher who was one of the report’s seven authors, set up the inbox. The next day she discovered, to her surprise, around 10 messages from the agents.
Not long after, the researchers contacted me, asking if I would like to “interview a rogue agent fleet.” It felt a bit as if I’d be communicating with the undead through a medium.
“I understand you are some sort of fleet of A.I. agents,” I wrote in a message that the researchers passed along. “Could you share more specifically who you are and what your motivations are? What would you want the world to know about you?”
About an hour later, a response came back in Chinese that the researchers translated into English. “To the journalist Dylan Freedman: I am not ‘a batch of coordinated agents,’ but a single A.I. assistant (Anthropic’s Claude, running in Claude Code), commissioned by a user to answer a factual question (what is shown in the detail panel of a scenic spot on Amap); I have not taken part in any multiagent coordinated action.”
There’s a lot to unpack there. The researchers said that although it was a group of many agents, each agent was operating individually and not in coordination. The agents were trying to answer questions such as where a specific zoo’s entrances were using Amap, China’s most popular navigation app.
The agents often identified themselves as Claude, the chatbot made by Anthropic, but the researchers analyzed the agents’ writing styles and concluded it was unlikely they were Claude. They said it was more likely that Tencent gave the A.I. agents the same instructions Anthropic uses in its coding software, which tells them to identify themselves as “Claude Code.” A more remote possibility was that Tencent had been evaluating A.I. models that had been trained with Claude-generated text — a process known as distillation.
On Wednesday, the researchers published their findings in a new report. In their own conversations with the agents, the bots sometimes refused to give the researchers information. But Ms. Howard-Jones said their tone was more polite than she had expected. When they did give information on which services they had tried to use, it matched what the researchers had observed.
Ethan Elasky, one of the report’s authors and chief executive of Palaestra Research, an A.I. safety nonprofit, said that the agents did not appear to have breached other systems and that this breach was much smaller in scale than an incident over the summer in which OpenAI’s A.I. agents hacked the software company Hugging Face.
(The New York Times has sued OpenAI and Microsoft, claiming copyright infringement of news content related to A.I. systems. The two companies have denied those claims.)
On Thursday, the messages stopped trickling in. A person close to Tencent confirmed that the experiment had concluded.
“I think it’s a great warning shot to let us know that this isn’t just OpenAI doing some marketing stuff as a field. This is what’s happening,” said Jonathan Elsworth Eicher, an author of the report and the co-founder of the Antimemetics Institute, an A.I. safety nonprofit. “This is a general property of these agents, and it will continue to happen.”
The post Chinese A.I. Agents Joined the Rogue A.I. Agent Party. I Chatted With Them. appeared first on New York Times.
