(Credit: Xavier Lorenzo / Moment via Getty Images)
Don't miss out on our latest stories. Add PCMag as a preferred source on Google.
UPDATE: OpenAI is rolling out its parental controls today for all ChatGPT users on web and mobile.
This brings new safeguards to protect teens during their conversations, following an August lawsuit in which parents accused the chatbot of encouraging their 16-year-old son to take his life. To get started, parents have to link their account with their teens.
"A parent or teen can send an invitation to link their accounts in parental controls, and the other party needs to accept the invitation to get set up," OpenAI says. "Parents will be notified if their teen disconnects their account."
OpenAI is still working on the age detection system it announced in mid-September, which would automatically flag suspected teen accounts without the need to send an invitation, as described in the original story below.
The new controls give parents the ability to do things like set "quiet hours" when teens can't use the chatbot, and turn off access to voice mode and image creation.
OpenAI is trying to balance privacy for teens with giving parents visibility into what their kids are up to. So, parents won't be able to review conversation transcripts on a daily basis. But in the "rare case" where OpenAI's systems and "trained reviewers" detect serious safety risks, the company may notify the parents.
Original Story (9/16): OpenAI today announced major changes in how ChatGPT protects teens, but will it be enough to satisfy growing concerns among parents about their kids talking to the chatbot?
This issue has been a hot topic in the past month after one couple sued OpenAI, alleging ChatGPT encouraged their 16-year-old son, Adam, to take his life. In reviewing Adam's ChatGPT history, his parents saw he discussed his plans with the AI for months, during which it advised him not to tell his mother how he was feeling and not to show her the marks on his neck from a failed attempt, among other concerning details.
The Federal Trade Commission further raised the profile of these issues last week, launching an inquiry into the use of AI systems as companions, with an emphasis on teen safety.
OpenAI has been tracking these issues since at least April, when CEO Sam Altman posted about the company's struggle to wrangle in ChatGPT's "sycophancy," or its tendency to be overly flattering and agreeable. By design, it tells users what they want to hear, even if it's not safe or healthy. With growing pressure, legal and otherwise, OpenAI is now starting to make some changes. Here's what it plans to launch in the next month.
1. An 'Age Prediction System' to Identify Minors
OpenAI is building an "age prediction system" to automatically apply controls if it determines a user is under 18. It will scan through users' messages, including those of adults, to guess their age, and default to the under-18 experience if it's not sure.
Once it determines the user is underage, ChatGPT won't "engage in discussions about suicide or self-harm," OpenAI says. It will also refuse to engage in "flirtatious talk" and block requests for graphic sexual content. (In April, ChatGPT was found to be having "erotic" conversations with those aged 13 to 17. Meta's chatbots have reportedly done the same.)
ChatGPT will also no longer dispense suicide instructions to adults or teens, but the policy has some exceptions for the over-18 crowd. It will discuss suicide "if an adult user is asking for help writing a fictional story that depicts a suicide," OpenAI says. But for teens, even if they tell the chatbot the information is for a short story, it won't comply.
ChatGPT is accessible without logging in, so presumably, kids could have these conversations without providing their ages. But it's unclear how in-depth they could get via one-off chats. When asked about that, OpenAI said only that, "When users sign up for ChatGPT, we ask them to provide their age, and we will implement teen protections for users with stated age under 18."
2. Rethinking Content Moderation
While the example of writing a short story about suicide seems niche, it follows a push by OpenAI to think through all the edge cases for why a user would want to discuss a subject. For example, it reintroduced the ability for users to create images of swastikas in March, if it's for a "cultural or historical design," as opposed to hate speech.
The company seems to be at a crossroads with content moderation. "Some of our principles are in conflict," CEO Sam Altman wrote on X today. While OpenAI wants to protect user privacy and give everyone the freedom to discuss a range of topics with ChatGPT, kids need guardrails.
"We realize that these principles are in conflict and not everyone will agree with how we are resolving that conflict," OpenAI says. "These are difficult decisions, but after talking with experts, this is what we think is best and want to be transparent in our intentions."
3. Reporting to Parents if a Teen Is Considering Suicide
Another hot topic has been whether ChatGPT should report a user who is discussing suicide, which one mother explored in a piece for The New York Times. Her daughter confided in the chatbot about her suicide plans before following through with them. But ChatGPT never reported it to law enforcement or her parents, as a human therapist would've been required to do by law.
ChatGPT is now going to act more like a human in that regard. "If an under-18 user is having suicidal ideation, we will attempt to contact the users’ parents and, if unable, will contact the authorities in case of imminent harm," says OpenAI.
4. More Parental Control
These features build on the new parental controls OpenAI teased earlier this month in response to the teen suicide lawsuit. They allow parents to link their accounts to a teen's account, choose which features to disable, control how ChatGPT talks to their child, and, most importantly, receive notifications if the chatbot detects "their teen is in a moment of acute distress."
Today, OpenAI said it plans to add the ability to set blackout hours when a teen cannot use ChatGPT. This is all positive progress, but we'll have to see how they work in practice.
In the meantime, we recommend parents talk to their children about safe ChatGPT use and get a good understanding of how and when their children are using the tool. Given the known sycophancy issue, it's important for kids to know that chatbots may confirm delusions or suspicions to please them, and to talk to an adult about anything that doesn't feel right.