Iran, China and groups in Israel used artificial intelligence in novel ways in recent months to create first-of-their-kind influence campaigns on social media, ratcheting up concerns among U.S. officials and security researchers about the fast-evolving technology.
The countries and groups combined A.I. tactics to ramp up the speed and scale of online campaigns beyond what people could easily achieve and with minimal human interaction, U.S. officials and security researchers with knowledge of the efforts said.
First, the countries and groups turned to Chinese “open-source” A.I. models, which are increasingly powerful systems that are freely available to download and modify. Then they used those A.I. models to create “agents,” which are bots that can operate software and complete tasks on their own.
Once hundreds of those A.I. agents were let loose, they autonomously opened networks of fake accounts on platforms such as Instagram, Facebook, X and TikTok, the officials and security researchers said. The agents then rapidly seeded the fake accounts with false posts about politics and current events to sway opinions and inflame issues.
It was one of the first times that A.I. agents backed by Chinese models were used to handle every stage of influence campaigns, from the creation of the fake accounts to coordinating their messaging online, said the U.S. officials, who were not authorized to speak publicly on sensitive issues. In most of the cases, the safeguards that would normally stop an A.I. model from creating a fake account or manipulating discussions online were turned off by those operating the systems, they added.
The “agentic” campaigns were crude, but caught the attention of U.S. intelligence, major tech companies and security experts, the people with knowledge of the campaigns said. These groups then worked to detect who was behind the efforts and their methods, finding that the Chinese and the Iranian campaigns were state-backed, while two Israeli campaigns were traced to private companies.
The campaigns have stoked fears over how publicly available A.I. models can be used to carry out online influence efforts essentially on their own. Some U.S. officials said they worried similar moves could be made to target U.S. voters ahead of the midterm elections in November.
To varying degrees, Russia, Iran and China have all targeted American voters with online influence campaigns during past election cycles, and each has used at least rudimentary A.I. Some U.S. officials and lawmakers have worried that recent A.I. advances could make election influence campaigns in the United States and globally more frequent and easier to orchestrate.
Agent-led campaigns will probably become more common as the technology continues to improve, said Kyle Crichton, a fellow at the Georgetown Center for Security and Emerging Technology, who researches A.I. and cybersecurity. “Agents can be fairly sophisticated, and so in addition to just scale, they have an ability to blend in and look more like a human user,” he said.
Debate over A.I. safety has escalated in recent weeks, fueled by the disclosure in July that A.I. systems from OpenAI had gone rogue and hacked into the start-up Hugging Face. Since then, A.I. leaders including Dario Amodei, the chief executive of Anthropic, have called for a slowdown in the development of the technology so that proper guardrails can be built. Other tech executives have said that no slowdown is needed and that Anthropic is trying to cement its own power.
In published reports this year, TikTok and Meta, which owns Facebook and Instagram, said that A.I. was being used to create accounts and content on their platforms. In one report last month, Meta said it had seen a “technically significant development” of bad actors experimenting with A.I. The company identified Iranian and Israeli campaigns that used A.I., without providing details.
A.I. was now “embedded within automated systems that can produce, adapt, and distribute content with minimal human intervention,” Meta said in its report.
X did not respond to requests for comment.
A.I. has long been used to create misleading videos and text and to generate convincing photos. But the agentic campaigns stood out, security researchers said.
Agents made by companies like OpenAI have been misused before, including by Iran, as early as August 2024, but open A.I. models allow for a new level of autonomy, said Kyle Chan, a researcher at Brookings. Unlike closed A.I. models from companies like OpenAI and Anthropic, where the underlying code is not disclosed, open A.I. models can be run on private hardware, meaning no company can intervene to shut the systems down.
The state-backed Iranian campaign was especially concerning, the officials and security researchers said. Iran targeted U.S. audiences, with the agent-generated fake accounts posing as everyday Americans who lived in major cities. The accounts tagged journalists and politicians in comments that spread popular memes and anti-Republican Party views, the people said. Nearly 80,000 people followed the A.I.-created accounts, which were active in the first half of the year.
Axios previously reported on some aspects of the Iranian network.
Intelligence and security officials did not provide more details on the Chinese campaign, other than to say it experimented with the same A.I. tactics.
Of the two agentic campaigns that were traced back to companies in Israel, one was generated over the summer by IntelEye, a Tel Aviv-based firm founded in 2023 by alumni of the NSO Group, a spyware firm that was blacklisted by the United States, according to the two people with knowledge of the campaign.
A U.S. security official briefed on IntelEye’s campaign said the company’s activities came to the attention of social media companies and U.S. intelligence earlier this year. The campaign used A.I. agents that created thousands of accounts on Facebook, Instagram, TikTok and X; the accounts responded to posts and comments discussing next month’s national elections in Israel. The accounts also posted content that both supported and opposed Israeli Prime Minister Benjamin Netanyahu, the official said.
Maor Sellek, an IntelEye co-founder, said in an interview that his company used DeepSeek, an open-source Chinese A.I. model, to control thousands of accounts across social media, but said it was an experiment to test the safety of A.I. models.
“IntelEye does not operate influence campaigns,” Mr. Sellek said. “Our work with DeepSeek was conducted for defensive research and testing, to understand how models with insufficient safeguards could be misused and to improve detection of that activity.”
Mr. Sellek said IntelEye bought 10,000 social media accounts for A.I. agents to control; roughly 1,000 were operational. The accounts left comments under videos or posts and tried to engage people in conversation, with little success, he said. The A.I. model chose the prompts and the topics the accounts posted on, he added.
In hindsight, he said, IntelEye should have told the social media companies it was conducting the tests.
“This appears to have been a coordinated attempt to create inauthentic accounts across several platforms,” a Meta spokesman said. “Only a small percentage of these accounts were on our platforms and we had already proactively removed the vast majority of them through our automated detection systems.”
In its report last month, Meta noted a separate campaign run by a “commercial operation based in Israel.” It did not name the company or provide other information, but said the campaign used A.I. in a “fairly sophisticated setup.”
The post Iran and China Create First-of-Their-Kind Autonomous A.I. Influence Campaigns appeared first on New York Times.
