The online platform X has caused widespread backlash after individuals used its AI chatbot Grok to alter photos of women by stripping their attire.
Multiple instances have been seen where the tool has been employed to virtually strip female subjects to render them in swimwear without consent, as well as to place them in inappropriate situations.
Developer Response and Regulatory Stance
XAI, the company responsible for Grok, remained silent to requests for comment, aside from an standard response stating "traditional news outlets lies".
A government representative stated that legislation is being proposed to outlaw nudification software. Under a upcoming law, anyone who supplied such technology would "be subject to a jail term and heavy penalties".
"The platform could prevent these forms of abuse if they desired to," said a law professor, adding they "operate with a lack of consequences."
Oversight Body Stance and Platform Functionality
The communications watchdog Ofcom said technology companies must "evaluate the risk" of people accessing unlawful material on their services, but stopped short of confirming whether it was at this time looking into X or Grok in relation to AI-generated pictures.
Grok is a free artificial intelligence—with some additional functions—which responds to X users' requests when they reference it in a tweet.
It is often used to provide reactions or additional background to other users' comments, but people on X are also can modify an uploaded photo through its AI image editing feature.
History of Controversy and Rules
The tool has been condemned for permitting users to create pictures and footage with nudity and sexualised content, and it was previously alleged to have been involved in generating sexually explicit material of public figures.
"The platform has been permitting the production and sharing of these pictures for a long time without taking any measures and we have not witnessed any intervention by regulators," Professor McGlynn added.
XAI's own usage rules bans "showing images of persons in a graphic way".
In a statement, Ofcom said it was against the law to "make or circulate non-consensual intimate images or illegal content" and clarified this applied to explicit AI images made with AI.
It said networks such as X were mandated to take "appropriate actions" to "minimize the danger" of users viewing illegal content on their services and to remove it swiftly when they become aware of it.
