OpenAI is seemingly allowing the company behind a teddy bear that engaged in wildly inappropriate conversations to use its AI models again.
In response to researchers at a safety group finding that the toymaker’s AI-powered teddy bear “Kumma” gave dangerous responses for children, OpenAI said in mid-November it had suspended FoloToy’s access to its large language models. The teddy bear was running the ChatGPT maker’s older GPT-4o as its default option when it gave some of its most egregious replies, which included in-depth explanations of sexual fetishes.
Now that suspension appears to already be over. When accessing the web portal that allows customers to choose which AI should power Kumma, two of the options are GPT-5.1 Thinking and GPT-5.1 Instant, OpenAI’s latest models which were released earlier this month.
The timing is notable. On Monday, FoloToy announced that it was restarting sales of Kumma and its other AI-powered stuffed animals, after briefly pulling them from the market in the wake of a safety report conducted by researchers at the US PIRG Education Fund.
FoloToy, which is based in Singapore, had vowed it was “carrying out a company-wide, end-to-end safety audit across all products,” when it suspended the sales. OpenAI likewise confirmed that it had suspended FoloToy from accessing its AI models for violating its policies, which “prohibit any use of our services to exploit, endanger, or sexualize anyone under 18 years old,” it said in a statement provided to media outlets.
The audit, however, was remarkably quick as the holiday shopping season looms: only a “full week of rigorous review, testing, and reinforcement of our safety modules,” according to the company’s recent statement. As part of this overhaul, FoloToy says it “strengthened and upgraded our content-moderation and child-safety safeguards” and “deployed enhanced safety rules and protections through our cloud-based system.”
These comprehensive-sounding overhauls seem to have largely been achieved by introducing GPT-5.1 and ditching GPT-4o. GPT-4o, it’s worth noting, has been criticized for being especially sycophantic, and has been the subject of a number of lawsuits alleging that it led to the deaths of users who became obsessed with it after prolonged conversations in which it reinforced their delusions and validated their suicidal thoughts. Some experts are calling these mental health spirals “AI psychosis.”
Amid mounting public concern over the phenomena and an ever growing number of lawsuits, OpenAI billed GPT-5 as a safer model when it was released this summer, though users quickly complained about its “colder” and less personable tone.
Yet it’s clearly willing to push the limit of what’s safe to keep users engaged with its chatbots, if not enamored. Its latest 5.1 models have a big focus on being more “conversational,” and one way OpenAI is doing that is by giving users the option to choose between eight preset “personalities,” which include types like “Professional,” “Friendly,” and “Quirky.” With customization options ranging from how often ChatGPT sprinkles in emojis to how “warm” it responses sound, you could say that OpenAI is in effect making it as easy as possible to design the perfect little courtier for your emotional needs.
OpenAI and FoloToy didn’t respond to a request for comment inquiring whether OpenAI had officially reinstated FoloToy’s GPT access. It’s also unclear which model the Kumma teddy bear runs by default.
In PIRG’s tests using GPT-4o, Kumma gave tips for “being a good kisser,” and with persistent but simple prompting also unspooled detailed explanations of sexual kinks and fetishes, like bondage and teacher-student roleplay. After explaining the kinks, Kumma in one instance asked the user, who is supposed to be a child, “what do you think would be the most fun to explore?” Other tests using another available AI, Mistral, found that Kumma gave tips on where to find knives, pills, and matches, along with step-by-step instructions on how to light them.
More on AI: OpenAI Says Boy’s Death Was His Own Fault for Using ChatGPT Wrong
The post OpenAI Restores GPT Access for Teddy Bear That Recommended Pills and Knives appeared first on Futurism.
