ResearchOpenAI

OpenAI limits weapon content and protects teens in ChatGPT

Original title: Community safety report and new protections for teens

Plain language

What it is: OpenAI announced new safety measures for ChatGPT, including a new Community Safeguards Report and stronger protections for teenagers. The updates focus on preventing violence and restricting content related to weapons and past mass attacks.
How it works: ChatGPT will restrict detailed information about past mass attacks and weapons. The system can also detect warning signs of violence in conversations, automatically end chats that fixate on harm, and apply stricter limits for users predicted to be under 18.
Where it is available: These updates apply to ChatGPT, including ChatGPT for Teens, and will begin rolling out in the next few weeks.
Example: If a teenager asks ChatGPT for practical information about weapons or security systems, the system will restrict the response even if the question appears harmless.

What it is: OpenAI is introducing a Community Safeguards Report and expanding ChatGPT safety policies to mitigate violence-related risks. The updates include stricter boundaries on queries about mass attacks, automated chat termination for violent fixation, and enhanced age-prediction safeguards.
How it works: The system uses short, factual summaries of previous warning signs to adjust its responses in subsequent chats. It also employs an age-prediction system analyzing behavioral signals to automatically apply teen-specific restrictions and can programmatically end sessions or pause accounts.
Where it is available: The features apply to ChatGPT and ChatGPT for Teens, with a rollout starting in the next few weeks.
Example: When a user's recent chat history shows warning signs of potential violence, ChatGPT will automatically respond more conservatively to queries that could be misused.

The headline and explanation on this page are based on OpenAI's post. The full details are in the original.

Community safety report and new protections for teens