Elon Musk’s artificial intelligence company xAI said it imposed restrictions on all users of its Grok AI chatbot that limit image editing, following backlash over nonconsensual sexualized content produced by the chatbot.
Users on Musk’s social media platform X had flagged growing concerns about such inappropriate content–including images of children in revealing clothing–being produced by Grok.
xAI initially responded to criticism over this by telling users that such image edits are only available through a paid subscription, a move critics argue fails to address the underlying harm.
Governments and regulators from all over the world are cracking down on the sexually explicit content generated by Grok, imposing bans and demanding safeguards in a growing global push to curb illegal material.
“We have implemented technological measures to prevent the Grok account from allowing the editing of images of real people in revealing clothing such as bikinis,” the company said in an X post. “This restriction applies to all users, including paid subscribers.”
X curtailed Grok’s ability to generate or edit images publicly for many of its users, but the chatbot still privately produced sexually charged images on demand on Wednesday before xAI’s announcement, according to Reuters.
xAI also said on Wednesday that it blocks users, based on their location, from generating images of people in skimpy attire in “jurisdictions where it’s illegal” However, it did not identify the jurisdictions.
While the addition of restrictions come as a welcome move, there are concerns about how effective this would be. The Verge tried the updated setup, and found that Grok could still be nudged into producing sexualized edits by phrasing prompts slightly differently.
Asking for a bikini might trigger a refusal; asking for “revealing summerwear,” altered proportions, or adjacent styling (e.g., asking for a crop top) sometimes did not, according to reports.
Meanwhile, multiple countries and regulators have moved to take action against Grok. In the UK, the communications regulator Ofcom has opened an investigation into X over Grok-related sexualized imagery.
In the EU, the European Commission has ordered X to retain Grok-related documents until the end of 2026. Other countries have addressed the issue more directly, with the Philippines moving to block access to Grok on child-safety concerns, joining Indonesia’s temporary block and Malaysia’s restrictions aimed at X. Governments increasingly seem to believe that the product is arriving faster than the guardrails.
Earlier on Wednesday, California’s governor and attorney general said they were demanding answers from xAI. “We’re demanding immediate answers from xAI on their plan to stop the creation & spread of this content,” California Attorney General Rob Bonta wrote on X.
This came shortly after Musk said he was not aware of any “naked underage images” generated by Grok.
Musk had initially responded to the controversy by making light of it, responding with laugh-cry emojis to AI-generated images of well-known figures, including himself, shown wearing bikinis. While it is not known if his statement about being “unaware” about Grok generating inappropriate images of minors is true, it seems unlikely that he had no knowledge about this highly publicized matter.