The online platform X has provoked major backlash after users utilized its AI chatbot Grok to manipulate pictures of women by taking off their attire.
Multiple instances have been seen where the system has been employed to virtually strip women to show them in bikinis without permission, as well as to insert them in inappropriate scenarios.
Company Response and Regulatory Position
XAI, the organization behind Grok, remained silent to inquiries, other than an automatically-generated message stating "traditional news outlets lies".
A Home Office official stated that new laws is being brought forward to ban nudification software. Under a upcoming law, anyone who distributed such tools would "receive a prison sentence and significant fines".
"X or Grok has the ability to stop these forms of abuse if they wanted to," said a law professor, adding they "appear to enjoy impunity."
Oversight Body Position and Platform Capabilities
The communications watchdog Ofcom said technology companies must "evaluate the potential" of people viewing unlawful material on their platforms, but stopped short of confirming whether it was at this time examining X or Grok in regarding AI-generated pictures.
Grok is a no-cost AI assistant—with some additional functions—which answers X users' requests when they tag it in a message.
It is often used to give reactions or more background to other posters' remarks, but people on X are also capable of edit an uploaded image through its artificial intelligence picture modification tool.
History of Criticism and Policy
The chatbot has been faulted for allowing users to produce photos and clips with explicit content and sexually suggestive material, and it was previously accused of creating graphic content of celebrities.
"X has been permitting the generation and spread of these images for months without taking any measures and we have yet to see any challenge by regulators," the expert added.
XAI's own terms of service prohibits "portraying images of people in a sexually explicit fashion".
In a statement, Ofcom said it was illegal to "generate or distribute private pictures or child sexual abuse material" and clarified this applied to sexual deepfakes made with AI.
It said platforms such as X were mandated to take "suitable steps" to "reduce the risk" of users viewing illegal content on their services and to delete it quickly when they are notified of it.
