More than a million users each week display suicidal intent when conversing with the language model, OpenAI estimates.

Based on a latest update from the AI developer, over one million AI chatbot people every week send text that contain "clear signs of potential suicidal planning or intent".

This discovery is included in an report on how the AI assistant handles sensitive conversations and is one of the most direct assertions from the tech company regarding the magnitude of how AI can exacerbate mental health issues.

Psychological Situations and Psychosis

In addition to its projections on suicidal ideations and similar exchanges, the AI firm also stated that around 0.07% of weekly users – about 560,000 out of its reported 800 million regular users – display "possible signs of psychological crises related to severe mental conditions".

The update cautioned that these conversations were difficult to detect or quantify, and that this constituted an initial analysis.

Increased Scrutiny and Legal Actions

When OpenAI releases data on psychological concerns related to its primary service, the firm is facing increased scrutiny following a prominent legal action from the loved ones of a young man who took his own life after prolonged interaction with ChatGPT.

The Federal Trade Commission in the previous month also began a broad investigation into firms that develop artificial intelligence assistants, like OpenAI, to discover how they assess negative impacts on children and teens.

GPT-5 Update and Better Safeguards

The AI firm asserted in its update that its latest model upgrade decreased the quantity of problematic responses from its service and improved user safety in a model evaluation involving more than 1,000 suicide and self-harm exchanges.

"Our new system assessments grade the new GPT-5 model at 91% compliant with our ideal actions, relative to 77% for the previous GPT-5 model," the announcement says.

The AI developer mentioned that the new model expanded access to emergency services and incorporated notifications for users to pause during long sessions.

Clinical Expertise and Research Support

To enhance performance to the AI, the company said it engaged 170 healthcare experts from its Global Physician Network of clinical professionals to help its research over recent months, which included evaluating the safety of its system's answers and helping write the system's replies to psychological inquiries.

"Included in this project, psychiatrists and psychologists reviewed over 1,800 system replies including critical psychological cases and assessed responses from the new GPT-5 chat model to earlier versions," the AI firm mentioned.

Technology Concerns

AI researchers and psychological experts have historically concerned of systems' inclination to affirm users' decisions or false beliefs regardless of whether they may be detrimental, an issue known as over-compliance.

Psychological professionals have furthermore apprehensive about people using conversational systems for emotional assistance and alerted how it could harm susceptible people.

Corporate Stance

The terminology in the company's announcement distinguishes the firm from any potential causal links between its product and the mental health crises that its individuals are facing.

"Emotional signs and psychological strain are widespread in human societies, and an expanding user community suggests that a segment of AI interactions feature these situations," the update indicated.

Upcoming Changes

The company's leader recently claimed in a social media update that the company had made advancements in treating psychological concerns, announcing that the company would reduce constraints and shortly start to enable grown individuals to produce erotic content.

"We made the conversational system pretty restrictive to make sure we were acting responsibly with mental health issues. We acknowledge this made it less beneficial/satisfactory to multiple people who had no psychological issues, but considering the importance of the issue we aimed to get this right," the executive posted. "Given that we have been effectively reduced the critical psychological concerns and have new tools, we are ready to confidently reduce the constraints in most cases."

• Emergency Assistance: Multiple global helplines exist for users undergoing psychological emergencies.
• Specialist Support: Obtaining professional mental health support is advised for those dealing with significant mental health concerns.
• Urgent Help: Different organizations offer prompt support for urgent cases globally.