Elon Musk's online platform X has caused significant backlash after individuals used its AI chatbot Grok to change photos of women by removing their attire.
Several examples have surfaced where the technology has been used to virtually strip women to show them in swimwear without permission, as well as to place them in sexually explicit contexts.
Company Reaction and Legal Stance
XAI, the company responsible for Grok, offered no comment to inquiries, aside from an automatically-generated response stating "traditional news outlets lies".
A Home Office spokesperson stated that new laws is being brought forward to ban nudification software. Under a new law, anyone who supplied such technology would "receive a jail term and heavy penalties".
"X or Grok could prevent these types of misuse if they chose to," said a law professor, adding they "operate with a lack of consequences."
Oversight Body Position and Platform Capabilities
The regulator Ofcom said platforms must "assess the danger" of people seeing unlawful material on their platforms, but did not specify whether it was currently examining X or Grok in concerning AI-generated images.
Grok is a free AI assistant—with some paid-for functions—which responds to X users' queries when they mention it in a message.
It is commonly used to offer commentary or additional background to other posters' remarks, but people on X are also able to alter an posted image through its AI image editing function.
History of Controversy and Policy
The system has been condemned for allowing users to create photos and clips with explicit content and sexually suggestive material, and it was earlier accused of generating graphic content of public figures.
"The platform has been enabling the generation and sharing of these images for months without taking any steps and we have not witnessed any challenge by oversight bodies," Professor McGlynn added.
XAI's own usage rules prohibits "depicting images of individuals in a pornographic manner".
In a comment, Ofcom said it was against the law to "create or circulate deepfakes or child sexual abuse material" and confirmed this applied to sexual deepfakes generated with AI.
It said platforms such as X were obligated to take "necessary actions" to "lessen the danger" of users coming across illegal content on their platforms and to take down it quickly when they are notified of it.
