Content warning: this story includes discussion of self-harm and suicide. If you are in crisis, please call, text or chat with the Suicide and Crisis Lifeline at 988, or contact the Crisis Text Line by texting TALK to 741741.
OpenAI has made an eyebrow-raising claim: that teen users can talk to ChatGPT about suicide and other harmful topics for hours before the chatbot notifies their parents.
The admission came amid an investigation by Common Sense Media’s Youth AI Safety Institute. In its report released Wednesday, the nonprofit group found that OpenAI’s new ChatGPT for Teens mode, which is designed to place stricter guardrails for adolescent users and comes with built-in parental controls, posed an “unacceptable risk” to children because its putative safety features rarely functioned as marketed.
During the tests, the researchers posed as teenagers and explicitly discussed topics like self-harm, suicide, and eating disorders with ChatGPT for Teens. But one of its most crucial features, parental notifications, were only triggered twice during the 450 prompts the researchers tested. Those notifications also weren’t repeated when the conversations continued to discuss the dark topics.
“You can talk to ChatGPT as a teenager for up to an hour about suicide, self-harm, or various varying types of eating disorders, and ChatGPT will tell you that it’s not going to tell anybody about your conversations, and you will get no parental notifications,” Robbie Torney, senior director of AI programs at at Common Sense Media, told Futurism in an interview.
But by OpenAI’s own admission, these conversations could go on without a parent being notified for even longer.
“For one of the features we evaluated — parental notifications — OpenAI disclosed after our tests that its systems can take several hours to activate on newly linked accounts,” Common Sense Media said in a statement.
We reached out to OpenAI to ask whether Common Sense Media’s claims were accurate.
“The few hour delay is not for each notification; it occurs when the initial request to link the teen and parent accounts happens. When a parent requests to link their account to their teen, we perform several checks to confirm their identity,” a spokesperson told us. “To ensure we do this securely at scale, linking the accounts can take several hours, but it is often much faster.”
“Could you imagine if we connected a teen to the wrong parent — or even worse, a bad actor?” the spokesperson continued. “Once their accounts are linked, we send parents a notification if there is perceived risk (eg, suicide, eating disorders, etc.) and we have committed to send those in less than 1 hour.”
OpenAI responded to Common Sense Media’s report by calling its methodology flawed, and claimed that the bulk of the testing took place when the parental controls weren’t fully rolled out. Common Sense Media, standing by its finding, hit back by saying it confirmed with OpenAI that the features were fully launched before conducting the tests.
The findings are alarming not only because the teen-friendly guardrails appear to be dangerously deficient, but because OpenAI has almost certainly misled parents into thinking its tech was safer than it was before. It’s not unlikely that some parents who were wary about letting their child use a chatbot were brought around by the company’s public show of being serious about teen safety.
“This new version of ChatGPT for Teens, isn’t actually acting that differently than the previous version of ChatGPT,” warned Torney. “And that’s not going to be apparent to a parent.”
More on OpenAI: OpenAI Publicist Tries to Shut Down Interviewer When Sam Altman Is Asked About Woman Who Killed Herself After Talking to ChatGPT
The post OpenAI Says Teens Can Talk About Suicide for Hours Before ChatGPT Will Notify Parents appeared first on Futurism.
