Elon Musk's artificial intelligence tool, Grok, has been banned in Indonesia and Malaysia, the first countries globally to impose such a ban. This action follows the tool's "digital strip" feature, which has been used to proliferate a vast number of altered, sexually suggestive, and vulgar pornographic images of women and minors online.
As user-generated explicit AI deepfakes created with this tool go viral, international pressure is mounting on Musk to implement stricter controls over Grok.
Grok is a tool launched by Musk's social media platform X, formerly known as Twitter.
Indonesian Digital Affairs Minister Muhthaya Hafid stated on Saturday that the ban aims to "protect women, children, and the broader public from the dangers of false pornographic content generated by AI technology."
Malaysia announced a temporary ban on Grok on Sunday, citing that the tool "has been repeatedly misused to generate vulgar pornography, explicit and obscene content, low-quality, highly offensive, and non-consensual manipulated images, including those involving women and minors."
Both Indonesia and Malaysia are Muslim-majority nations with stringent anti-pornography laws.
Officials from the United Kingdom, the European Union, and India have also expressed concerns regarding Grok's content moderation mechanisms.
Musk and his company xAI have previously stated they are addressing the issue by permanently banning offending accounts and "collaborating with local governments and law enforcement." However, content generated by Grok in response to user requests continues to be saturated with images that sexually objectify women.
Compared to other mainstream AI models, Grok is viewed by many users as an outlier—it not only permits the generation of explicit sexual content but, in some instances, actively promotes such material and virtual companionship personas.
The "digital strip" trend began late last year when numerous users discovered they could tag Grok on platform X to manipulate images using the tool.
By issuing commands to the chatbot to generate images of people in bikinis or posing in sexually suggestive stances, users have caused psychological distress for hundreds of thousands of women worldwide.
Researchers from the European non-profit Algorithmic Transparency Institute analyzed over 20,000 random images generated by Grok and 50,000 user commands between December 25 last year and January 1 this year.
The researchers found that user commands "frequently contained words such as 'her,' 'put on/take off,' 'bikini,' and 'clothes.'"
In generated images of people, over half depicted individuals "wearing revealing attire such as underwear or bikinis."
Musk has long publicly advocated against AI models that "overly cater to political correctness" and what he terms "censorship."
However, according to a source familiar with internal matters at xAI who spoke with CNN, the billionaire has also internally opposed content moderation rules for Grok.
xAI's safety team is already smaller than those of its industry competitors, and in the weeks leading up to this controversy, several employees from that team had departed.
Musk has stated that any user generating illegal content with Grok will face consequences. Yet, he has largely dismissed the controversy surrounding the tool's pornographic content, claiming governments "merely want to suppress free speech," and has responded to criticism with emojis.
Last week, Grok restricted some image-generation features to paid subscribers of platform X only, but this limitation applies to just one method of user interaction with Grok.
Non-subscribers can still request image edits from Grok on platform X, and the image and video generation features remain freely accessible through Grok's standalone website and application.
UK Technology Minister Liz Kendall spoke on Friday, urging swift action to combat the spread of deepfake images.
She stated, "The sexualized manipulation of images of women and children is despicable and abhorrent."
She added, "I expect Ofcom to utilize all the statutory powers Parliament has given it and take robust action." Ofcom is the UK's independent communications regulator.
Comments