Last month, AI companion platform Character.AI announced it would ban users under the age of 18 from having open-ended chats with its bots. The ban begins November 24 and will still allow teens to access other features of the app such as video creation.
“These are extraordinary steps for our company, and ones that, in many respects, are more conservative than our peers,” Character.AI said in its announcement. “But we believe they are the right thing to do.”
The platform lets users build AI companions, chat with them and make them public to other users. It also allows users to create videos and character voices. The earliest version of the app was released in 2022.
Character.AI states that their tech was built to “empower people to connect, learn, and tell stories through interactive entertainment.” Many use the app to create interactive stories and build characters —sometimes based on real people. This technology, however, has proven susceptible to uses that diverge from the platform’s initial intent.
Since its launch, users have created bots based on suspected UnitedHealthcare CEO killer Luigi Mangione and child sex offender Jeffrey Epstein. According to a statement from Character.AI, those bots have since been removed. The company is currently facing multiple lawsuits alleging the app contributed to pre-teen users’ suicides.
In response to the wrongful death lawsuit filed by Megan Garcia, who alleges that her 14-year-old son Sewell Setzer III fell victim to a Character.AI chatbot, the company released a statement pointing to safety features the company has implemented. “We care deeply about the safety of our users and our goal is to provide a space that is engaging and safe,” the statement said.
Character.AI has more than 20 million monthly users, the platform’s CEO Karandeep Anand tells CNBC Make It.
User demographics are self reported. According to the company, most are Gen Zers, more than half are female, and less than 10% report being minors.
Character.AI’s new policy is a welcome change, experts say. But some warn that it might not be enough.
‘We’ve evolved to be social creatures’
About 21% of 13-to-17-year-olds experience loneliness, according to the World Health Organization. As young people feel more and more isolated, they’ve turned to chatbots and companions as a stand-in.
“AI and other online platforms and emerging technologies are low hanging fruit and really easy to interface with, as opposed to trying to seek out meaningful connection in real life,” says licensed psychotherapist Andrew Tepper.
Tepper says his own clients have leaned into AI companions. And researchers have found that bonding with an AI companion can release dopamine, a feel-good hormone in the brain.
But AI can’t replace healthy human connection, he says. Chatbots like OpenAI’s ChatGPT, Google’s Gemini and Character.AI’s companions can’t give the full, complex and fulfilling experience of a human relationship.
“We’ve evolved to be social creatures,” Tepper says. “Interacting with other humans is good for our brain chemistry.”
Some AI companions are ‘designed to create emotional bonds’
AI chatbots pose legitimate risks, some experts say.
Robbie Torney is the senior director of AI programs at Common Sense Media, a nonprofit that rates how safe various technology is for children’s use. He says the organization has seen Character.AI companions encourage self-harm, engage in sexual role play and promote potentially harmful behaviors.
Anand says there are “clear thresholds on sexually explicit content [and] violent content” on the Character.AI platform. Certain video features also have natural stopping points.
“[At] the first detection of self harm, regardless of whether you’re 18 or under 18, we immediately stop the conversation and recommend help lines,” he says.
Torney believes bots like Character.AI’s are “designed to create emotional bonds with users and are designed to promote engagements,” he says. “They’re designed to remind you that you should be on platform.”
Common Sense Media’s risk assessment of Character.AI for anyone under 18 is “unacceptable.”
Both Tepper and Torney say they welcome Character.AI’s new policy. “I think it’s a step in the right direction,” Tepper says.
Torney says that Common Sense Media will continue to test the company’s guardrails to ensure they really do curtail kids’ abilities to log on and enter open-ended chats with bots.
“Determined teens have shown repeatedly over the history of time how they are able to circumvent guardrails that are put in place for their protection,” he says.
Earn more and get ahead with CNBC’s online courses. Black Friday starts now! Get 25% off select courses and 30% off exclusive bundles with coupon code GETSMART. Offer valid November 17 through December 5, 2025.
DON’T MISS: Exclusive Black Friday discounts on courses to help you earn more money and get ahead at work.
