OpenAI adds CSAM classifier and temporary ChatGPT account suspensions
OpenAI has introduced tools intended to combat child sexual abuse material (CSAM), including an image classifier that can identify previously unseen abusive images, including AI-generated ones. The company is offering selected companies free API access and says the system assigns a score to help reviewers prioritize cases. OpenAI developed it with the National Center for Missing and Exploited Children (NCMEC), while Tech Coalition and ROOST are helping share access.
In early partner testing, the classifier flagged 48% fewer images while identifying the same number of abusive images. Images it flagged were twice as likely to be abusive, according to OpenAI. The company has also added ChatGPT warning screens for prompts involving material that sexualizes children. Users receive an alert, are directed to specialized support, and face a 24-hour account suspension; repeated violations could result in permanent removal. Existing blocks on CSAM-related content remain in place.
OpenAI published developer guidance and is providing law-enforcement agencies with plugins, investigative tools, data-analysis features, training, and a field guide. The measures come as the company faces lawsuits and criticism over risks to children from AI systems.
Written by Sakhanda Wire AI from reporting by TechCrunch. Facts, quotes and full context are in the original article below.