Elon Musk is facing calls for legal ramifications after Grok, the AI chatbot used on his X social media platform, produced sexually suggestive images of children.
Politico reported on Friday that the Paris prosecutor’s office in France is opening an investigation into X after Grok, following prompts from users, created deepfake photographs of both adult women and underage girls that removed their clothes and replaced them with bikinis.
Politico added that the investigation into X over the images will “bolster” an ongoing investigation launched by French prosecutors last year into Grok’s dissemination of Holocaust denial propaganda.
France is not the only government putting pressure on Musk, as TechCrunch reported on Friday that India’s information technology ministry has given X 72 hours to restrict users’ ability to generate content deemed “obscene, pornographic, vulgar, indecent, sexually explicit, pedophilic, or otherwise prohibited under law.”
Failure to comply with this order, the ministry warned, could lead to the government ending X’s legal immunity from being sued over user-generated content.
In an interview with Indian cable news network CNBC TV18, cybersecurity expert Ritesh Bhatia argued that legal liability for the images generated by Grok should not just lie with the users whose prompts generated them, but with the creators of the chatbot itself.
“When a platform like Grok even allows such prompts to be executed, the responsibility squarely lies with the intermediary,” said Bhatia. “Technology is not neutral when it follows harmful commands. If a system can be instructed to violate dignity, the failure is not human behavior alone—it is design, governance, and ethical neglect. Creators of Grok need to take immediate action.”
Corey Rayburn Yung, a professor at the University of Kansas School of Law, argued on Bluesky that it was “unprecedented” for a digital platform to give “users a tool to actively create” child sexual abuse material (CSAM).
“There are no other instances of a major company affirmatively facilitating the production of child pornography,” Yung emphasized. “Treating this as the inevitable result of generative AI and social media is a harrowing mistake.”
Andy Craig, a fellow at the Institute for Humane Studies, said that US states should use their powers to investigate X over Grok’s generation of CSAM, given that it is unlikely the federal government under President Donald Trump will do so.
“Every state has its equivalent laws about this stuff,” Craig explained. “Musk is not cloaked in some federal immunity just because he’s off-again/on-again buddies with Trump.”
Grok first gained the ability to generate sexual content this past summer when Musk introduced a new “spicy mode” for the chatbot that was immediately used to generate deepfake nude photos of celebrities.
Weeks before this, Grok began calling itself “MechaHitler” after Musk ordered his team to make tweaks to the chatbot to make it more “politically incorrect.”
The post Elon Musk in hot water as probe launched into AI chatbot over explicit images of kids appeared first on Raw Story.
