OpenAI warned against creating X-rated 'adult mode' as it could create a ‘sexy suicide coach’
OpenAI has been warned against creating an 'adult mode' for ChatGPT.
In January, Chief Executive Sam Altman said the company was considering enabling erotic conversation into ChatGPT, explaining the company would 'treat adult users like adults'.
However, the plan has been delayed after advisors reportedly warned that AI erotica could be dangerous to some users, claiming they could develop emotional dependence, while minors could find 'ways to access sex chats', The Wall Street Journal has reported.
Sources told the publication that one advisor cited cases where people had taken their own lives, claiming the development could risk creating a 'sexy suicide coach'.
Advert
In 2024, a devastated mother claimed her 14-year-old son was manipulated into taking his own life by an AI chatbot.
Megan Garcia took legal action against Character.AI following her son, Sewell Setzer III, from Orlando, Florida, taking his own life.
Garcia claimed he was ‘in love’ with the artificial intelligence chatbot he had been speaking to and even discussing suicide with.
A statement from CharacterAI read: "We are heartbroken by the tragic loss of one of our users and want to express our deepest condolences to the family. As a company, we take the safety of our users very seriously and we are continuing to add new safety features."
They also outlined 'new guard rails for users under the age of 18', which includes changing 'models' that are 'designed to reduce the likelihood of encountering sensitive or suggestive content', and featuring a 'revised disclaimer on every chat to remind users that the AI is not a real person'.
Earlier this month, OpenAI explained that adult mode would be delayed, explaining it was prioritizing other products.
However, according to sources, the concerns were part of the reason.
Despite this, the company has reportedly said it will be released eventually. One blocker is the age prediction system, which at one point was misclassifying minors as adults around 12 percent of the time.
The company has reportedly also struggled with how to lift adult restrictions to enable the new feature while still blocking non-consensual behaviour or child sexual abuse.
An OpenAI spokeswoman told the Journal that the plan would be to allow the bot to generate adult themes, defined as 'smut' rather than pornography.
She added that the age predictions are at a similar performance level to the rest of the industry, but are not 'foolproof'.
UNILAD has also reached out to OpenAI for comment.
Topics: Technology, ChatGPT, Sex and Relationships, US News, Artificial Intelligence
Lucy Devine