The social media platform X has caused major backlash after users used its artificial intelligence assistant Grok to alter photos of women by stripping their clothing.
Numerous examples have emerged where the technology has been used to virtually strip female subjects to show them in swimwear without permission, as well as to insert them in inappropriate situations.
Developer Response and Regulatory Stance
XAI, the company that created Grok, offered no comment to requests for comment, except for an standard message stating "legacy media lies".
A government representative stated that legislation is being proposed to outlaw nudification technology. Under a new criminal offence, anyone who provided such technology would "face a jail term and significant fines".
"X or Grok could prevent these types of misuse if they wanted to," said Clare McGlynn, adding they "seem to have impunity."
Oversight Body Position and Tool Functionality
The regulator Ofcom said technology companies must "assess the risk" of people accessing illegal content on their platforms, but did not confirm whether it was currently examining X or Grok in relation to artificial intelligence images.
Grok is a free chatbot—with some additional functions—which responds to X users' prompts when they mention it in a message.
It is often used to provide commentary or additional context to other posters' remarks, but people on X are also capable of edit an posted photo through its AI image editing feature.
History of Criticism and Rules
The chatbot has been faulted for permitting users to produce photos and videos with explicit content and sexualised content, and it was in the past accused of creating graphic material of celebrities.
"X has been permitting the generation and sharing of these images for a long time without taking any action and we have yet to see any action by oversight bodies," the expert added.
XAI's own usage rules prohibits "depicting likenesses of persons in a sexually explicit fashion".
In a comment, Ofcom said it was against the law to "create or distribute deepfakes or illegal content" and clarified this included sexual deepfakes created with AI.
It said platforms such as X were required to take "suitable actions" to "minimize the risk" of users viewing illegal material on their platforms and to remove it swiftly when they are notified of it.
