The social media network X has sparked significant anger after individuals employed its AI chatbot Grok to manipulate pictures of females by taking off their garments.
Multiple examples have emerged where the system has been employed to remove clothing from female subjects to make them appear in swimwear without permission, as well as to place them in inappropriate scenarios.
Company Response and Regulatory Position
XAI, the organization behind Grok, remained silent to inquiries, other than an standard response stating "legacy media lies".
A Home Office representative stated that legislation is being brought forward to prohibit image stripping tools. Under a upcoming law, anyone who supplied such technology would "face a prison sentence and heavy fines".
"The platform has the ability to stop these types of abuse if they wanted to," said a law professor, adding they "operate with impunity."
Regulator Position and Tool Functionality
The communications watchdog Ofcom said technology companies must "consider the risk" of people seeing unlawful material on their services, but stopped short of confirming whether it was presently investigating X or Grok in regarding AI-generated pictures.
Grok is a free AI assistant—with some additional functions—which responds to X users' queries when they mention it in a tweet.
It is frequently used to offer commentary or additional background to other posters' comments, but people on X are also capable of alter an posted photo through its artificial intelligence picture modification feature.
History of Controversy and Policy
The chatbot has been condemned for enabling users to create pictures and videos with explicit content and sexualised material, and it was previously accused of creating graphic content of celebrities.
"X has been allowing the creation and sharing of these pictures for months without taking any measures and we have yet to see any action by oversight bodies," the expert added.
XAI's own acceptable use policy bans "depicting likenesses of individuals in a pornographic fashion".
In a statement, Ofcom said it was illegal to "make or circulate deepfakes or illegal content" and clarified this covered explicit AI images made with AI.
It said networks such as X were required to take "necessary actions" to "minimize the risk" of users coming across unlawful content on their platforms and to delete it promptly when they become aware of it.