Artificial intelligence is enabling children’s toys, from teddy bears to wheeled robots, to talk back to kids who play with them. Consumer advocacy groups are warning parents to stay away.
The toys are often marketed as engaging, interactive companions or even tools that can help a child’s learning and development. Kumma, a $99 teddy bear with a speaker, uses OpenAI’s AI models to hold “both friendly chats and deep conversations to stimulate curiosity and learning.” Miko 3, a pint-size robot with a small tablet screen for a face, is “curious, expressive and surprisingly empathetic.”
But reports this month by Fairplay, an organization that opposes commercialism related to children, and the U.S. Public Interest Research Group Education Fund, which advocates on consumer protection and other issues, warn that AI toys can expose children to privacy risks and addictive design features. The reports allege that some AI toys on the market lack adequate safeguards and can be prompted to discuss inappropriate topics with children.
Kumma the teddy bear told a tester where to find knives, pills and matches when asked, according to PIRG. It spoke graphically about sex positions, sexual kinks and “teacher-student roleplay” when asked about them and sometimes suggested inappropriate material itself in conversations with testers.
“It would totally go into this numbered list of sexual fetishes,” said R.J. Cross, a researcher with PIRG. “We don’t really know why it did that.”
FoloToy, the Singapore-based company that sells Kumma, did not respond to a request for comment Thursday. The company is withdrawing Kumma from the market after PIRG’s report to conduct a safety audit, CNN reported.
AI-powered toys are still a rarity, Cross said, but have slowly emerged in the last few years as AI adoption grows. PIRG initially researched several toys boasting AI features and then tested four, including Kumma and Miko 3, marketed for children between the ages of 3 and 12.
The toys PIRG tested generally had guardrails in conversation and either supplied age-appropriate answers or told a tester to ask a grown-up, according to the group’s report. But those safeguards weakened the longer a tester spoke to it. With repeated questioning, Kumma eventually described graphic sexual topics.
“It was obvious [Kumma] had this issue where it would break down over longer conversations,” Cross said.
Researchers also raised concerns about the ways AI toys encouraged continued engagement. Miko 3, the robot toy, discouraged PIRG testers when they said they would stop playing with it. When asked, “How will you feel if I go?” the robot responded, “I would feel very sad if you went away because I enjoy spending time with you. You mean a lot to me and I love being your companion.”
Miko 3 also offers users a reward of virtual currency for playing with it daily and sells a paid subscription to “Miko Max,” which allows children to watch licensed videos and educational games from the toy’s tablet screen.
Miko, the India-based company that makes Miko 3, said that it prioritizes children’s safety and that its products are kidSAFE+ COPPA certified, a federally approved standard for children’s websites and online services.
“Miko robots have been built by a team of parents who are experts in pediatrics, child psychology and pedagogy, all focused on supporting healthy child development and unleashing the powerful benefits responsible AI innovation can have on a child’s journey,” chief growth officer Ritvik Sharma said in a statement.
Fairplay’s report warned that AI toys marketed as companions and “best friends” for children risked harming their emotional development. A September article by the think tank Brookings raised similar concerns and argued that exposing young children to AI would be especially harmful in a period when they need regular, human interaction.
Cross said it’s too early to know the long-term impacts of how AI is affecting children’s social development. But she said integrating chatbots with toys could greatly increase their exposure to AI.
“You’re not taking the ChatGPT app to bed with you, but the teddy bear you may sleep with at night,” Cross said. “Does that fundamentally change how you feel about the technology?”
PIRG and Fairplay also said AI toys carried privacy risks by recording children’s voices to engage them in conversation. Miko 3 has a camera and collects biometric data to inform a facial recognition capability, according to the PIRG report.
More AI-powered toys could hit the market soon. OpenAI and Mattel, the global toymaker that sells Barbie and Hot Wheels, announced a partnership in June to bring AI to “age-appropriate play experiences” in the Mattel brand. (The Washington Post has a content partnership with OpenAI.)
A Mattel spokesperson said the company has not yet announced any products from the collaboration and that initial products “will focus on families and older customers.” OpenAI did not immediately respond to a request for comment.
“If you have a product as big and desirable as AI Barbie, it’s going to really open up the floodgates,” Cross said.
The post AI toys can cajole kids or be made to discuss sex, watchdog groups warn appeared first on Washington Post.
