The social media platform X has provoked widespread outrage after people used its AI chatbot Grok to change images of women by taking off their garments.
Several cases have been seen where the tool has been used to digitally undress female subjects to render them in bikinis without consent, as well as to insert them in sexualized situations.
Company Response and Legal Stance
XAI, the firm behind Grok, offered no comment to questions, other than an automatically-generated response stating "legacy media lies".
A Home Office spokesperson stated that new laws is being introduced to ban image stripping software. Under a upcoming criminal offence, anyone who provided such technology would "be subject to a jail term and substantial fines".
"The platform has the ability to stop these types of misuse if they chose to," said a law professor, adding they "operate with impunity."
Oversight Body Stance and Platform Capabilities
The regulator Ofcom said platforms must "consider the potential" of people seeing illegal content on their platforms, but did not specify whether it was currently looking into X or Grok in regarding artificial intelligence pictures.
Grok is a free AI assistant—with some premium functions—which responds to X users' prompts when they mention it in a post.
It is often used to offer reactions or additional background to other posters' remarks, but people on X are also capable of alter an posted photo through its artificial intelligence picture modification feature.
Background of Controversy and Rules
The tool has been faulted for enabling users to generate photos and clips with nudity and sexually suggestive material, and it was previously alleged to have been involved in creating inappropriate content of celebrities.
"The platform has been enabling the production and distribution of these images for months without taking any steps and we have not witnessed any challenge by regulators," the expert added.
XAI's own acceptable use policy prohibits "depicting likenesses of individuals in a graphic fashion".
In a comment, Ofcom said it was against the law to "make or circulate deepfakes or illegal content" and clarified this covered sexual deepfakes generated with AI.
It said services such as X were obligated to take "suitable measures" to "reduce the danger" of users viewing illegal content on their platforms and to delete it promptly when they become aware of it.
