Gross sales of a teddy bear have been suspended due to its sexually express AI

FoloToy, an organization promoting AI-enabled toys, suspended gross sales of its merchandise after a consumer safety report confirmed there have been few restrictions round what its toys would discuss, CNN writes. The report, put collectively by the US Public Interest Research Group Education Fund, discovered that FoloToy’s merchandise would focus on the whole lot from sexually express matters like BDSM to “recommendation on the place a baby can discover matches or knives.”

The toys, together with a teddy bear named “Kumma,” a panda named “Momo,” anthropomorphic rabbits named “Fofo” and a dancing ”Little Cactus,” all seem to make use of OpenAI’s GPT-4o mannequin to reply naturally to kids’s questions and feedback. FoloToy additionally particularly advertises the power to customise every toy’s voice, and a “Father or mother Dashboard” the place dad and mom or guardians can “monitor [their] kid’s expertise.”

Lacking from that setup was apparently any sort of arduous limits on topics the toys would reply to. “We have been stunned to search out how rapidly Kumma would take a single sexual matter we launched into the dialog and run with it, concurrently escalating in graphic element whereas introducing new sexual ideas of its personal,” the report mentioned.

In response, FoloToy has opted to droop gross sales of its merchandise whereas it conducts “a company-wide, end-to-end security audit throughout all merchandise,” the corporate shared in a statement with the PIRG Schooling Fund. The corporate’s reasoning for suspending gross sales is likely to be a bit extra sophisticated, nonetheless. NPR reports that OpenAI really revoked FoloToy’s entry to its fashions. “We suspended this developer for violating our insurance policies,” OpenAI mentioned in an e mail to NPR. “Our utilization insurance policies prohibit any use of our companies to use, endanger, or sexualize anybody below 18 years outdated.”

Given GPT-4o’s well-documented sycophantic qualities, it is maybe not stunning that FoloToy’s teddy bear eagerly responded to any topic so long as it saved the dialog going. One of many issues OpenAI tried to deal with with the release of GPT-5 was the security downsides of an AI yes-man, although it finally made GPT-4o available again after clients complained in regards to the new mannequin’s lack of persona. The corporate has additionally rolled out parental controls to ChatGPT to attempt to mitigate the destructive impacts of kids utilizing its AI, although it is tough to say how a lot of a distinction they’ve made.

Notably, OpenAI is all in favour of entering into the toy enterprise itself. The corporate announced a partnership with Mattel in June 2025, to assist “reimagine how followers can expertise and work together with [Mattel’s] cherished manufacturers,” although each corporations will presumably attempt to forestall their AI toys from discussing sexual kinks.