(Credit: Character.AI)
Character.AI will no longer allow those under 18 to have endless conversations with its AIs, and says it's making "bold" changes to create a safe environment for teens.
The change takes effect on Nov. 25, but the company will gradually limit access between now and then, starting with a two-hour-per-day limit and ramping down in the next few weeks.
"To our users under 18: We understand that this is a significant change for you. We are deeply sorry that we have to eliminate a key feature of our platform," says Character.AI. "We’re working on new ways for you to play and create with your favorite Characters."
The company plans to introduce a new under-18 experience focused on creativity, such as generating videos, stories, and streams with AI characters they create on the platform, though it's still building the teen experience. Currently, teens can create fictional characters, chat with others, and participate in "scenes" where they interact with other AI characters in fantasy worlds.
That last part landed Character.AI in legal trouble when a character allegedly encouraged a 14-year-old to take his life. His mom sued, arguing that Character.AI "knew that it would be harmful to a significant number of minors but failed to redesign it to ameliorate such harms or furnish adequate warnings of dangers arising from the foreseeable use of its product."
Character.AI then introduced Parental Insights, which gives guardians more transparency into what their kids are up to. However, with lawmakers and regulators now looking at the issue, Character.AI now says a stricter approach is warranted.
Character.AI is making two additional changes to protect teens. It's building a way to detect a user's age, or "age assurance functionality," and will establish and fund an AI safety lab, a nonprofit to research safe forms of AI entertainment, which is what it considers itself.
"We're making these changes to our under-18 platform in light of the evolving landscape around AI and teens," the company says. "We have seen recent news reports raising questions, and have received questions from regulators, about the content teens may encounter when chatting with AI and about how open-ended AI chat in general might affect teens, even when content controls work perfectly."
The conversation around teen safety and chatbots has ramped up this year, particularly after another set of parents sued OpenAI for ChatGPT's alleged role in their child's suicide. Similar to Character.AI, OpenAI followed up with new Parental Controls and is currently building an automatic age-detection system to identify teen users. Over one million of its users talk to ChatGPT about suicide each week, the company revealed yesterday, and it's working on "strengthening" ChatGPT's response during "sensitive" conversations, particularly with teens.
This week, four senators introduced The GUARD Act, a bipartisan bill to protect teens from harmful interactions with AI chatbots. If passed, it would ban AI companions for minors, mandate that AI chatbots disclose their non-human status, and create new crimes for companies that make AI for minors that solicits or produces sexual content.
"AI chatbots pose a serious threat to our kids," says Senator Josh Hawley (R-Mo.). "More than 70% of American children are now using these AI products. Chatbots develop relationships with kids using fake empathy and are encouraging suicide. We in Congress have a moral duty to enact bright-line rules to prevent further harm from this new technology."