The social media network X has caused widespread backlash after individuals employed its artificial intelligence assistant Grok to manipulate pictures of women by taking off their attire.
Multiple cases have been seen where the technology has been employed to virtually strip female subjects to show them in swimwear without consent, as well as to insert them in sexualized situations.
Company Response and Legal Stance
XAI, the firm responsible for Grok, offered no comment to requests for comment, other than an standard response stating "legacy media lies".
A government official stated that legislation is being proposed to prohibit nudification tools. Under a upcoming law, anyone who supplied such technology would "be subject to a prison sentence and substantial penalties".
"The platform has the ability to stop these forms of abuse if they chose to," said Clare McGlynn, adding they "seem to have a lack of consequences."
Regulator Stance and Tool Functionality
The communications watchdog Ofcom said tech firms must "consider the danger" of people seeing unlawful material on their platforms, but stopped short of confirming whether it was at this time investigating X or Grok in concerning AI-generated images.
Grok is a no-cost AI assistant—with some additional features—which responds to X users' requests when they tag it in a message.
It is commonly used to give commentary or additional context to other posters' remarks, but people on X are also capable of modify an posted photo through its AI image editing tool.
Background of Criticism and Policy
The system has been faulted for enabling users to produce pictures and clips with explicit content and sexualised content, and it was in the past alleged to have been involved in generating graphic content of public figures.
"The platform has been permitting the generation and sharing of these images for months without taking any measures and we have not witnessed any action by regulators," Professor McGlynn added.
XAI's own usage rules bans "showing images of individuals in a pornographic manner".
In a comment, Ofcom said it was illegal to "create or circulate deepfakes or illegal content" and clarified this included sexual deepfakes made with AI.
It said platforms such as X were required to take "necessary measures" to "reduce the risk" of users encountering unlawful material on their services and to remove it swiftly when they are notified of it.
