The online network X has sparked major backlash after users employed its AI chatbot Grok to alter images of females by taking off their attire.
Several cases have been seen where the system has been employed to digitally undress women to show them in swimwear non-consensually, as well as to insert them in sexualized scenarios.
Developer Response and Regulatory Stance
XAI, the firm behind Grok, offered no comment to inquiries, other than an automatically-generated message stating "traditional news outlets lies".
A government spokesperson stated that legislation is being brought forward to outlaw nudification technology. Under a new law, anyone who supplied such tools would "face a jail term and significant penalties".
"The platform is capable of halting these types of abuse if they chose to," said Clare McGlynn, adding they "appear to enjoy impunity."
Oversight Body Position and Platform Functionality
The regulator Ofcom said technology companies must "consider the danger" of people seeing unlawful material on their networks, but did not confirm whether it was presently investigating X or Grok in concerning artificial intelligence images.
Grok is a no-cost AI assistant—with some additional functions—which answers X users' queries when they reference it in a post.
It is often used to offer reactions or additional background to other users' remarks, but people on X are also can alter an uploaded picture through its artificial intelligence picture modification tool.
Background of Controversy and Policy
The system has been criticised for allowing users to create pictures and footage with explicit content and sexually suggestive material, and it was previously accused of creating sexually explicit content of celebrities.
"X has been allowing the production and distribution of these images for a long time without taking any measures and we have not seen any action by oversight bodies," the expert added.
XAI's own acceptable use policy bans "showing images of individuals in a graphic fashion".
In a statement, Ofcom said it was illegal to "create or circulate deepfakes or illegal content" and clarified this included sexual deepfakes made with AI.
It said platforms such as X were required to take "suitable steps" to "minimize the danger" of users coming across illegal content on their services and to take down it quickly when they become aware of it.
