The social media network X has sparked widespread backlash after individuals used its AI chatbot Grok to manipulate pictures of females by taking off their clothing.
Several examples have been seen where the system has been used to remove clothing from women to show them in swimwear without permission, as well as to insert them in sexualized scenarios.
Company Response and Legal Stance
XAI, the firm that created Grok, did not respond to questions, other than an standard response stating "legacy media lies".
A Home Office official stated that legislation is being introduced to prohibit image stripping technology. Under a proposed law, anyone who provided such tools would "face a prison sentence and substantial fines".
"The platform is capable of halting these types of abuse if they desired to," said a law professor, adding they "appear to enjoy impunity."
Oversight Body Position and Platform Capabilities
The regulator Ofcom said tech firms must "evaluate the danger" of people accessing illegal content on their platforms, but stopped short of confirming whether it was presently looking into X or Grok in relation to artificial intelligence images.
Grok is a free chatbot—with some paid-for features—which responds to X users' prompts when they reference it in a post.
It is frequently used to offer commentary or further context to other posters' remarks, but people on X are also can edit an uploaded image through its artificial intelligence picture modification feature.
History of Controversy and Policy
The chatbot has been faulted for enabling users to create photos and clips with nudity and sexually suggestive content, and it was previously accused of making sexually explicit content of celebrities.
"The platform has been enabling the creation and spread of these images for months without taking any steps and we have not witnessed any action by regulators," Professor McGlynn added.
XAI's own usage rules prohibits "portraying images of individuals in a graphic manner".
In a comment, Ofcom said it was unlawful to "create or share deepfakes or illegal content" and confirmed this covered explicit AI images made with AI.
It said platforms such as X were obligated to take "appropriate measures" to "reduce the risk" of users coming across illegal material on their platforms and to delete it swiftly when they are notified of it.
