The online network X has provoked widespread backlash after people employed its AI chatbot Grok to alter pictures of women by removing their garments.
Several cases have emerged where the system has been used to remove clothing from female subjects to render them in swimwear non-consensually, as well as to place them in sexually explicit scenarios.
Company Response and Regulatory Position
XAI, the organization responsible for Grok, did not respond to inquiries, aside from an automatically-generated message stating "legacy media lies".
A Home Office representative stated that new laws is being introduced to ban nudification software. Under a new law, anyone who distributed such tools would "face a jail term and heavy fines".
"X or Grok is capable of halting these forms of abuse if they wanted to," said a law professor, adding they "operate with a lack of consequences."
Oversight Body Stance and Platform Capabilities
The regulator Ofcom said platforms must "assess the potential" of people seeing unlawful material on their services, but stopped short of confirming whether it was currently investigating X or Grok in regarding artificial intelligence images.
Grok is a no-cost artificial intelligence—with some premium functions—which reacts to X users' requests when they reference it in a message.
It is often used to offer reactions or additional context to other posters' comments, but people on X are also able to modify an uploaded image through its AI image editing tool.
History of Criticism and Rules
The tool has been criticised for enabling users to produce images and videos with explicit content and sexualised material, and it was previously alleged to have been involved in generating graphic material of celebrities.
"The platform has been permitting the creation and spread of these pictures for months without taking any steps and we have yet to see any intervention by oversight bodies," the expert added.
XAI's own usage rules forbids "showing likenesses of individuals in a graphic fashion".
In a statement, Ofcom said it was illegal to "generate or distribute deepfakes or child sexual abuse material" and stated this included sexual deepfakes made with AI.
It said platforms such as X were obligated to take "necessary measures" to "minimize the danger" of users coming across unlawful content on their services and to take down it swiftly when they are notified of it.
