Instagram on Friday unveiled safety features for teenagers who use its artificial intelligence chatbots amid growing concerns over how the chatbots are affecting young people’s mental health.
The features, which will be rolled out early next year, would give parents more control over how teenagers use Instagram’s “A.I. characters,” which have fictional personalities that users can message with as they would other human accounts.
Parents would be able to block their children from having conversations with certain A.I. characters, and Instagram would send them summaries of their children’s chats, the company said. Instagram would also limit chatbot conversations on topics like self-harm, eating disorders and romance, while allowing “age-appropriate topics” like education, sports and hobbies.
“We hope today’s updates bring parents some peace of mind that their teens can make the most of all the benefits A.I. offers, with the right guardrails and oversight in place,” the company said in a blog post. The post was signed by Adam Mosseri, the head of Instagram, and Alexandr Wang, the chief A.I. officer of Meta, which owns Instagram.
A.I. chatbots, which can generate responses in chats in seemingly humanlike ways, have been under scrutiny for how they affect teens who spend hours a day confiding in them. They have been blamed for driving some children to suicide and sending some adults into delusional spirals.
Meta is not the only company contending with how to make its chatbots safer. Last month, OpenAI, the A.I. start-up behind ChatGPT, announced new teen safety features such as parental controls.
But Meta’s chatbots — which have personalities like schoolteacher, McDonald’s cashier and fortune teller — have faced particular criticism for having sexual conversations with underage users. In August, lawmakers began investigating Meta after Reuters reported that its chatbots were allowed to have provocative conversations about race and medical disinformation.
The changes are Instagram’s latest major safety update aimed at teens. Last year, the app announced that teen accounts would be private by default, making it more difficult for outsiders to interact with them. On Tuesday, Instagram also said that the types of content that teenagers could see would be guided by the PG-13 rating system used by the film industry.
Meta, which also owns Facebook, WhatsApp and Messenger, has long faced questions over how its apps affect children. The company has promised to protect minors from inappropriate content since 2008, and is grappling with personal injury lawsuits in state and federal court that accuse it of harming young people with an addicting product.
Most of Instagram’s A.I. characters, which were introduced in 2023, are created by users and shown in the same part of the app as messages with humans. The characters have their own identities and can engage in voice calls in addition to text conversations. Sometimes the characters will message users first, unprompted, throughout the day.
Meta has set some limits on the types of conversations that users can have with its chatbots, but Instagram allows them to speak in different kinds of “seductive” voices, and many of the characters are romantically themed to act as girlfriends or boyfriends.
The chatbots, which can help keep users engaged, are part of Meta’s broader ambitions around A.I. Mark Zuckerberg, Meta’s chief executive, has said he envisions a future where A.I. creates much of the content people see on Instagram and Facebook. And, Mr. Zuckerberg has said, the chatbots will provide companionship.
“There are all these things that are better about physical connections when you can have them,” Mr. Zuckerberg said on a podcast interview in April. “But the reality is that people just don’t have as much connection as they want.”
Mr. Zuckerberg has been on a campaign this year to revamp Meta’s A.I. division, hiring Mr. Wang and spending millions of dollars to recruit dozens of top A.I. technologists from rivals such as OpenAI.
Eli Tan covers the technology industry for The Times from San Francisco.
The post Instagram Unveils Teen Safety Features for A.I. Chatbots appeared first on New York Times.
