(Credit: Jonathan Raa/NurPhoto via Getty Images)
Don't miss out on our latest stories. Add PCMag as a preferred source on Google.
California Governor Gavin Newsom on Monday signed a bill to protect users, especially minors, from the potential harms of AI companions. However, he also vetoed another AI bill that would've required chatbots to more strictly police what they discussed with kids.
Newsom approved SB 243, which requires AI companies to ensure that their chatbots clearly inform users that they are AI systems, not humans. The chatbots should also be trained to avoid sharing information regarding suicide or self-harm and redirect users engaging in such conversations to crisis management helplines. Plus, AI companies must submit annual reports on how they manage users with suicidal tendencies, effective July 1, 2027.
In August, the parents of a teenager named Adam Raine sued OpenAI after they found their son had conversations about suicide methods with ChatGPT before taking his life. The chatbot initially resisted, but Raine bypassed the safeguards by stating he needed the information for writing and world-building purposes. OpenAI has since rolled out new parental controls.
"By setting clear guardrails, California is helping shape a more responsible approach to AI development and deployment across the country," an OpenAI spokesperson tells CNBC.
Additionally, SB 243 requires AI chatbots to remind minors to take a break at least every three hours. (ChatGPT added break reminders in August.) The bots must also stop generating sexually explicit content for minors and engaging in sexual conversations with them. ChatGPT and Meta AI have allegedly both engaged in inappropriate discussions with minors.
"We can continue to lead in AI and technology, but we must do it responsibly — protecting our children every step of the way," Newsom said in a statement. "Our children’s safety is not for sale.”
Newsom, however, rejected AB 1064, which would have banned companies from making their chatbots available to minors unless they could guarantee that the chatbot would not discuss certain topics.
"While I strongly support the author's goal of establishing necessary safeguards for the safe use of AI by minors, AB 1064 imposes such broad restrictions on the use of conversational AI tools that it may unintentionally lead ot a total ban on the use of these products by minors," Newsom said in his veto note. "AI is already shaping the world, and it is imperative that adolescents learn how to safely interact with AI systems."
Newsom pledged to "develop a bill next year that ensures young people can use AI in a manner that is safe, age-appropriate, and in the best interests of children and their future."
As TechCrunch notes, California is now the first US state to regulate companion AI chatbots with SB 243. It's one of the several technology bills Newsom signed this week. He also approved an age-verification bill for devices and app stores, which goes into effect on Jan. 1, 2027. Apple and Google have already outlined how they will comply with such legislation.
Disclosure: Ziff Davis, PCMag's parent company, filed a lawsuit against OpenAI in April 2025, alleging it infringed Ziff Davis copyrights in training and operating its AI systems.