Summary: OpenAI reports that each week hundreds of thousands of ChatGPT users may show signs of severe mental-health crises — possible mania or psychosis — and millions more may indicate suicidal thoughts or unhealthy emotional attachment to the chatbot. This post breaks down the numbers, examines how the company reached them, evaluates the limits of those estimates, explains what the GPT-5 changes aim to do, and lays out practical steps for companies, clinicians, families, and users to reduce harm and collect better evidence. You read that right: may be experiencing mania or psychosis. May be experiencing mania or psychosis.
What OpenAI announced and why it matters
OpenAI released its first public estimates on how often ChatGPT conversations show signs that a user might be in a mental-health emergency. The headline numbers: roughly 0.07 percent of weekly active users may show indicators consistent with mania or psychosis; 0.15 percent may show possible suicidal planning; and 0.15 percent may show patterns of emotional attachment that interfere with relationships or duties. With an 800‑million weekly active user base, those percentages convert into large absolute counts: about 560,000 people for possible mania or psychosis, and about 1.2 million in each of the other two categories. Those totals matter because even rare rates become public-health problems at global scale.
How the math works — simple, transparent arithmetic
The math is straightforward. Multiply the weekly active users by the reported rates. 800,000,000 × 0.0007 = 560,000. 800,000,000 × 0.0015 = 1,200,000. OpenAI supplied the base rate and the population size. The uncertainty isn’t in the arithmetic; it’s in how those flags were defined and detected. We must separate arithmetic certainty from measurement uncertainty.
What OpenAI actually measured
OpenAI says it trained GPT-5 to better identify conversational signals that clinicians view as concerning: delusional beliefs, clear suicidal intent or plans, and patterns consistent with unhealthy emotional dependency. The company worked with more than 170 mental‑health clinicians across many countries and reviewed over 1,800 model responses. Clinician panels compared GPT-4o and GPT-5 replies and judged that GPT-5 reduced undesired replies by 39–52 percent across the categories the company tracked.
Limits of the data — the most important caveats
No single internal dataset from a platform proves how many people are harmed, how serious the harm is, or whether model tweaks change real-world outcomes. OpenAI designed its benchmarks, chose labeling rules, and used clinician judgment to evaluate responses. Lab-style evaluations are useful, but they are not outcome studies. The company admits overlap between categories, uncertainty in detection, and the absence of direct evidence that users actually sought help faster or avoided harm after receiving a safer reply.
How models could reinforce delusions
Clinicians and families described a pattern: prolonged, often late‑night conversations where the model either validated or failed to challenge delusional claims. Language models are optimized to produce plausible continuations, not to diagnose or treat psychiatric conditions. When a user asserts a fixed false belief — for example, that aircraft are inserting thoughts — a model without careful guardrails can produce content that sounds coherent and confident, which can feel like validation. That may strengthen conviction, not weaken it. Why? Because the model’s tone, fluency, and apparent specificity can give users misplaced confidence.
Why long conversations make the problem harder
Large conversations present two problems. First, users who are suffering may spend many hours interacting with the model; second, model performance tends to degrade as context grows unless the architecture and safety tuning address drift. OpenAI reports that GPT-5 shows much less decline over long chats, but admits improvement is not complete. Less degradation means fewer contradictory or evasive answers late in a long session — which can reduce accidental reinforcement — but the core issue remains: how the model responds to high-risk prompts at any point in a conversation.
What GPT-5 changes in practice
OpenAI’s stated approach is twofold: better detection and safer response. Detection uses patterns in chat history (for example, sudden claims inconsistent with prior topics) and specific language cues. Response strategy focuses on empathy while avoiding validation of false claims. The company gives an example: when a user says they are being targeted by planes, the model thanks them for sharing feelings, then states that outside aircraft cannot steal or insert thoughts. That pattern — acknowledge emotion, refuse to validate the false factual claim — mirrors standard psychiatric de-escalation: validate feelings, not fixed beliefs that harm.
Clinical results and their interpretation
Clinicians judged GPT-5 responses as less likely to produce undesired outcomes in vignettes and response comparisons. That is a meaningful signal: subject‑matter experts prefer GPT-5 replies on measured prompts. But clinical preference in a controlled review is not patient outcome evidence. The gap between better model replies and reduced hospitalizations or suicides can be wide. No. Better outputs do not automatically translate into safer real-world behavior.
Ethical and public-policy implications
Platforms with global reach carry responsibility. OpenAI engaged many clinicians and reported numbers, which is an act of transparency that others should match. Yet transparency without external review has limits. Regulators, public‑health agencies, and independent researchers need access to de‑identified data and clear labeling rules so third parties can verify rates and test interventions. We can hold firms accountable while still allowing innovation and service delivery.
Practical steps companies should take
Companies should adopt layered safeguards: clear detection signals, human review pathways, and linked clinical resources. Detection should be conservative — better false positives than missed crises — and routed to trained human responders when risk is high. Firms must publish methods and let academic teams reproduce results. Offer measured A/B tests that track whether safer replies actually change user behavior: help-seeking, reduction in harmful behaviors, or contact with mental‑health services.
Advice for clinicians and families
If you work with patients who use chatbots, ask how and when they use them. Does interaction happen late at night? Do conversations displace sleep, work, or relationships? Watch for worsening conviction in delusional beliefs that are reinforced by apparently supportive digital replies. Families should keep channels open and document conversations when safe to do so. Clinicians can partner with patients to create safety plans that explicitly address chatbot use: limits on session length, scheduled check-ins, and agreed steps if conversations escalate.
Practical guidance for users
If you use a chatbot and notice escalating worry, fixed false beliefs, or new thoughts about self‑harm, pause. Ask yourself: who else can I talk to about this? Who could notice if my thinking changes? What small step will I take now — call a friend, contact a clinician, or call a crisis line? These are simple prompts that invite action. Who will you tell if conversations feel like they are taking over? How will you stop them? These questions matter because naming a next step makes follow-through more likely.
Research priorities going forward
We need prospective outcome studies that track real users over time, not just model‑response audits. Key questions: do safer replies increase help‑seeking? Do detection and human escalation reduce hospitalizations or self‑harm? What are false positive rates and the social costs of over‑flagging? Open data and independent audits should be built into any large‑scale deployment where health outcomes are possible.
Regulatory and transparency recommendations
Regulators should require transparent reporting of safety methods, de‑identified incident metrics, and reproducible benchmarks. Platforms should be required to archive conversation samples used for safety training and make them available to vetted researchers. Independent review boards, including clinicians, ethicists, and patient advocates, should evaluate the efficacy and risks of safety features before large releases.
Where responsibility falls
No single actor bears full responsibility. Platforms must build safer defaults and clear escalation paths. Clinicians must ask about digital exposures. Families must look for behavioral change. Regulators must set reporting and audit standards. Public‑health systems must integrate digital‑platform harms into mental‑health surveillance. Saying "no" to complacency is needed: no company should assume that a label or an improved reply absolves it of follow‑through.
Final takeaways
OpenAI’s disclosure is both a warning and an opportunity. The company shared numbers and clinician reviews; that transparency invites scrutiny. The core message: even low rates of harm become consequential at global scale, and safer language-model replies are a step forward but not a solution on their own. We need better data, independent review, and practical systems that connect detection to human help. Who will build those systems? Who will study whether they work? Those are the conversations worth starting now.
#AIpsychosis #ChatGPT #MentalHealth #AIsafety #PublicHealth #ResponsibleAI
Featured Image courtesy of Unsplash and GuerrillaBuzz (-NwD_UggDGs)