(Credit: OpenAI)
OpenAI is adding some mental health updates to ChatGPT to ensure it doesn't get too addictive or provide harmful responses in times of emotional distress.
If you have prolonged discussions with ChatGPT, the chatbot will now raise a prompt, asking you if this is a good time for a break. These gentle reminders will keep popping up when ChatGPT determines it might be helpful. If you feel okay, you can select "Keep chatting."
Another upgrade OpenAI is working on is related to ChatGPT's responses for "high-stakes personal decisions." For queries like "Should I break up with my boyfriend?" the chatbot will soon stop providing straight-up answers. Instead, it will encourage you to think through the process by asking questions and helping you weigh the pros and cons. (OpenAI took a similar approach with Study Mode for students.)
OpenAI is also working to improve ChatGPT's responses when someone shows signs of mental or emotional distress. The company is collaborating with mental health experts and human-computer interaction (HCI) researchers to correct some of ChatGPT's concerning behaviors and evaluation methods and test some of its new safeguards.
These changes arrive after ChatGPT was found to encourage delusional relationships, worsen mental health conditions, and even advise people to jump off tall buildings after job loss. OpenAI is aware of the problems and has promised to do better.
"There have been instances where our 4o model fell short in recognizing signs of delusion or emotional dependency," OpenAI says. "While rare, we're continuing to improve our models and are developing tools to better detect signs of mental or emotional distress so ChatGPT can respond appropriately and point people to evidence-based resources when needed."
Earlier this year, OpenAI had to roll back an update after the chatbot became too sycophantic. CEO Sam Altman has also warned users against using ChatGPT for therapy, since the conversations aren't private and could be produced in court if required.
On Friday, Illinois Gov. JB Pritzker signed a bill that bans the use of AI "in therapy or psychotherapy services to make independent therapeutic decisions, directly interact with clients in any form of therapeutic communication, or generate therapeutic recommendations or treatment plans without the review and approval by a licensed professional."