A new flood of child sexual abuse material created by artificial intelligence is hitting a tipping point of realism, threatening to overwhelm the authorities.
Over the past two years, new A.I. technologies have made it easier for criminals to create explicit images and videos of children. Now, researchers at organizations including the Internet Watch Foundation and the National Center for Missing & Exploited Children are warning of a surge of new material this year that is nearly indistinguishable from actual abuse.
New data released Thursday from the Internet Watch Foundation, a British nonprofit that investigates and collects reports of child sexual abuse imagery, identified 1,286 A.I.-generated videos of child sexual abuse so far this year globally, compared with just two in the first half of 2024.
The videos have become smoother and more detailed, the organization’s analysts said, because of improvements in the technology and collaboration among groups on hard-to-reach parts of the internet called the dark web to produce them.
The rise of lifelike videos adds to an explosion of A.I.-produced child sexual abuse material, or CSAM. In the United States, the National Center for Missing & Exploited Children said it had received 485,000 reports of A.I.-generated CSAM, including stills and videos, in the first half of the year, compared with 67,000 for all of 2024.
“It’s a canary in the coal mine,” said Derek Ray-Hill, interim chief executive of the Internet Watch Foundation. The A.I.-generated content can contain images of real children alongside fake images, he said, adding, “There is an absolute tsunami we are seeing.”
The deluge of A.I. material threatens to make law enforcement’s job even harder. While still a tiny fraction of the total amount of child sexual abuse material found online, which tallied reports in the millions, the police have been inundated with requests to investigate A.I.-generated images, taking away from their pursuit of those engaging in child abuse.
Law enforcement authorities say federal laws against child sexual abuse material and obscenity cover A.I.-generated images, including content that is wholly created by the technology and do not contain real images of children.
Beyond federal statutes, state legislators have also raced to criminalize A.I.-generated depictions of child sexual abuse, enacting more than three dozen state laws in recent years. .
But courts are only just beginning to grapple with the legal implications, legal experts said.
The new technology stems from generative A.I., which exploded onto the scene with OpenAI’s introduction of ChatGPT in 2022. Soon after, companies introduced A.I. image and video generators, prompting law enforcement and child safety groups to warn about safety issues.
Much of the new A.I. content includes real imagery of child sexual abuse that is reused in new videos and still images. Some of the material uses photos of children scraped from school websites and social media. Images are typically shared among users in forums, via messaging on social media and other online platforms.
In December 2023, researchers at the Stanford Internet Observatory found hundreds of examples of child sexual abuse material in a data set used in an early version of the image generator Stable Diffusion. Stability AI, which runs Stable Diffusion, said it was not involved in the data training of the model studied by Stanford. It said an outside company had developed that version before Stability AI took over exclusive development of the image generator.
Only in recent months have A.I. tools become good enough to trick the human eye with an image or video, avoiding some of the previous giveaways like too many fingers on a hand, blurry backgrounds or jerky transitions between video frames.
The Internet Watch Foundation found examples last month of individuals in an underground web forum praising the latest technology, where they remarked on how realistic a new cache of A.I.-generated child sexual abuse videos were. They pointed out how the videos ran smoothly, contained detailed backgrounds with paintings on walls and furniture, and depicted multiple individuals engaged in violent and illegal acts against minors.
About 35 tech companies now report A.I.-generated images of child sexual abuse to the National Center for Missing & Exploited Children, said John Shehan, a senior official with the group, although some are uneven in their approach. The companies filing the most reports typically are more proactive in finding and reporting images of child sexual abuse, he said.
Amazon, which offers A.I. tools via its cloud computing service, reported 380,000 incidents of A.I.-generated child sexual abuse material in the first half of the year, which it took down. OpenAI reported 75,000 cases. Stability AI reported under 30.
Stability AI said it had introduced safeguards to enhance its safety standards and “is deeply committed to preventing the misuse of our technology, particularly in the creation and dissemination of harmful content, including CSAM.”
Amazon and OpenAI, when asked to comment, pointed to reports they posted online that explained their efforts to detect and report child sexual abuse material.
Some criminal networks are using A.I. to create sexually explicit images of minors and then blackmail the children, said a Department of Justice official, who requested anonymity to discuss private investigations. Other children use apps that take images of real people and disrobe them, creating what is known as a deepfake nude.
Although sexual abuse images containing real children are clearly illegal, the law is still evolving on materials generated fully by artificial intelligence, some legal scholars said.
In March, a Wisconsin man who was accused by the Justice Department of illegally creating, distributing and possessing fully synthetic images of child sexual abuse successfully challenged one of the charges against him on First Amendment grounds. Judge James Peterson of the U.S. District Court for the Western District of Wisconsin said that “the First Amendment generally protects the right to possess obscene material in the home” so long as it isn’t “actual child pornography.”
But the trial will move forward on the other charges, which relate to the production and distribution of 13,000 images created with an image generator. The man tried to share images with a minor on Instagram, which reported him, according to federal prosecutors.
“The Department of Justice views all forms of A.I.-generated CSAM as a serious and emerging threat,” said Matt Galeotti, head of the Justice Department’s criminal division.
Cecilia Kang reports on technology and regulatory policy for The Times from Washington. She has written about technology for over two decades.
The post A.I.-Generated Images of Child Sexual Abuse Are Flooding the Internet appeared first on New York Times.
