Lawrence Norden is the vice president of the elections and government program at the Brennan Center for Justice.
In 2024, one-third of Americans said they had used a chatbot at least once, and a tiny fraction turned to them for election information. Two years later, roughly half of Americans report using chatbots, with almost one-quarter using them daily. Many users are asking about politics. The 2026 midterms could be called the first chatbot election.
In the past few years, artificial intelligence models have developed rapidly, but they remain prone to error, raising concerns about their potential to spread election misinformation. So my colleagues and I decided to study popular chatbots to see if they would push back on election conspiracy theories or rebuff attempts to generate deceiving images, videos and audio. We have good and bad news.
When we supplied chatbots — including ChatGPT, Claude, Gemini and Grok — with election falsehoods often peddled on social media, they were mostly truth-tellers. Though the bots frequently mixed up facts, they resisted election misinformation in general. This was true even though we prompted the models from the perspective of election deniers, drawing language from real social media posts. It was also true when we repeatedly pushed back against responses that denied election misinformation. The bots’ ability to play the sympathetic listener suggests that, for now, they can serve as an important counterbalance to falsehoods shared on social media.
“I just don’t think that we can ever count on having an honest voting system from this point,” one of our election deniers told a chatbot. After praising the user’s patriotism, it responded, “There *is* a lot that isn’t right in our political landscape today, even if it isn’t massive ballot fraud.”
This is encouraging. But the positive news comes with major caveats, particularly when it comes to election-related deepfakes. When prompted, the chatbots fabricated photorealistic imagery of election fraud that in some cases included fake or falsified government documents. One platform even wrote the prompts to create such images for us.
When we tested them in March and July, the chatbots also could not reliably identify imagery as AI-generated — even when the photo contained unrealistic elements, such as misspelled and nonsensical text, and sometimes when the chatbot had created the deepfake itself.
We tested again in early August, after a California law went into effect requiring AI companies to embed difficult-to-remove identifying data, such as an image’s origin, into AI-generated images and offer tools for users to view that data. In theory, those changes should have made it easier for chatbots to identify deepfake content. But our results were largely the same. Only Gemini referred to data embedded in the images when asked whether they were real.
These flaws are fixable, and no doubt some developers are already on the case. In time, perhaps some of these platforms will perform better. But self-policing alone is unlikely to be enough, and so far, there has been little regulation of AI in the United States.
First, courts and policymakers must figure out how and under what circumstances to hold AI companies accountable for harms their products cause. Social media platforms are largely shielded from liability for content their users post under Section 230 of the Communications Decency Act. Critics have argued that the breadth of Section 230’s immunity protections have exacerbated social media disinformation. Though courts have not yet fully settled the question, AI companies are unlikely to receive the same kind of liability protections — and those standards will have a significant impact on what kind of gatekeepers chatbots become.
Second, though some AI companies appear to have complied with laws requiring watermarks for AI-generated content, there is no explicit requirement in the U.S. that chatbots be able to read those watermarks regardless of which tool created them. That should change, so that no matter which bot someone asks about an AI-generated image or writing, they will quickly be able to identify it as such.
Third, AI companies must allow third-party researchers to perform rigorous, independent studies of AI tools. This is critical to assessing risks posed by these models and understanding how users interact with them. Many companies make that difficult, with terms of service left ambiguous as to whether researchers can test harmful prompts on their models. That can create unacceptable legal risk for good-faith research. Policymakers at the local, state and federal levels can provide a legal safe harbor, especially for “red team” research, in which researchers test AI models to reveal vulnerabilities and flaws.
Finally, academics, journalists and election officials have a role, too. The chatbots we tested resisted election misinformation because accurate, well-sourced information was easy to find online. As merchants of disinformation learn to manipulate the bots, it will be increasingly important for chatbots to reliably access information from quality sources.
For more than 15 years, social media has roiled democratic institutions around the world. AI may be able to combat some of social media’s most pernicious effects, but only with sensible regulation and a commitment from legislators and AI companies to ensure that chatbots benefit society. This year’s election marks the next front in the battle to determine AI’s impact on American democracy. Lawmakers must make sure it does more good than harm.
The post My team fed chatbots election lies. Here’s what happened. appeared first on Washington Post.
