Frontier AI labs are seemingly in a race to melt the impressionable minds of underage users into a puddle.
For years now, we’ve watched individuals under the age of 18 become obsessed with social AI companions, an alarming trend that has already had catastrophic consequences, from chatbots encouraging them to engage in self-harm to even being implicated in a series of deaths. And that’s without getting into the growing wave of evidence that AI is undermining education in profound ways.
The vast majority of AI labs happen to be headquartered in California, a state whose governor is now trying to get ahead of the problem — while simultaneously providing the companies with a home, in optics that are dicey at best.
On Thursday, the state’s governor Gavin Newsom signed what his office called “landmark bipartisan legislation” to protect “children online and when using artificial intelligence.”
As part of the legislation, AI companies are required to develop “crisis protocols in the case of suicidal ideation, parental controls, and notifications should a child disable safety settings.” One of the new laws was nicknamed “Adam’s Law,” in a nod to 16-year-old Adam Raine, who took his life in April 2025 after extensive conversations with ChatGPT about his intent to die.
The law “will save lives and prevent other serious harms — and hopefully lead other states to enact similar protections,” Adam’s mother Maria Raine said in a statement during the bill’s signing ceremony on Thursday.
The legislation also prohibits “social media platforms from offering addictive features to users under 16,” directing them to “expand privacy protections for children.” Addictive features under the law include “autoplay videos and algorithmic feeds based on user history and profile.”
While it’s certainly a well-intentioned move, enforcing these new rules could prove difficult. The laws require AI chatbots to verify users’ ages, which is far easier said than done. It could also expose underage users to new risks. Just this month, hackers breached what appears to have been a major ID card verification service.
It’s a serious issue. We’ve already seen more than one seemingly ChatGPT-facilitated mass shooting. Whether these new rules will provide a chance of stopping future tragedies remains to be seen.
Meanwhile, the pressure on California-based AI companies is building fast. As of this month, OpenAI is facing over 50 consumer harm and wrongful death lawsuits.
Reactions to the latest rules varied significantly. Meta largely took issue with the new restrictions against showing underage users algorithmic feeds, saying they were necessary to allow them to “explore their interests safely,” as a spokesperson told the Associated Press. That’s despite Meta agreeing last month to pay up to $18 billion in damages after being found to have harmed children with its platforms.
Anthropic, on the other hand, officially backed Newsom’s bill, telling Politico in a statement that while “artificial intelligence holds extraordinary promise,” it “must be developed and deployed with meaningful safeguards to protect the public.”
OpenAI has also promised to improve ChatGPT safeguards, per CNBC, but didn’t expand on how.
More on AI regulation: Top Chinese Court Issues Sweeping Legal Crackdown on AI Deepfakes
The post California, Which Is Creating All the AI That’s Poisoning Children, Just Cracked Down on AI Use for Its Own Kids appeared first on Futurism.
