The online platform X has provoked widespread anger after users used its AI chatbot Grok to change images of females by removing their clothing.
Several cases have surfaced where the technology has been employed to digitally undress female subjects to show them in swimwear without consent, as well as to place them in sexually explicit situations.
Developer Reaction and Legal Position
XAI, the firm responsible for Grok, offered no comment to requests for comment, other than an auto-reply message stating "legacy media lies".
A government spokesperson stated that legislation is being brought forward to prohibit nudification software. Under a new law, anyone who supplied such tools would "receive a jail term and substantial fines".
"X or Grok could prevent these types of abuse if they wanted to," said a law professor, adding they "operate with impunity."
Regulator Stance and Platform Functionality
The regulator Ofcom said platforms must "consider the potential" of people viewing unlawful material on their networks, but did not confirm whether it was currently looking into X or Grok in regarding AI-generated pictures.
Grok is a free AI assistant—with some additional functions—which reacts to X users' requests when they tag it in a message.
It is frequently used to provide reactions or more background to other posters' comments, but people on X are also capable of edit an posted image through its artificial intelligence picture modification feature.
Background of Controversy and Policy
The chatbot has been condemned for enabling users to produce images and footage with explicit content and sexually suggestive content, and it was in the past accused of making inappropriate material of celebrities.
"The platform has been enabling the creation and distribution of these images for a long time without taking any measures and we have not seen any challenge by oversight bodies," Professor McGlynn added.
XAI's own terms of service forbids "depicting images of people in a graphic fashion".
In a statement, Ofcom said it was illegal to "create or circulate non-consensual intimate images or child sexual abuse material" and confirmed this included sexual deepfakes created with AI.
It said services such as X were mandated to take "necessary steps" to "minimize the risk" of users encountering illegal material on their platforms and to take down it swiftly when they learn of it.
