UK media regulator Ofcom has opened an investigation into X over reports that its Grok AI tool was used to create deepfake sexual images, including sexualised images of children. The probe tests the platform’s compliance with the country’s new online safety regime and could carry fines of up to 10% of global revenue if Ofcom finds breaches and the company fails to comply.
“There have been deeply concerning reports of the Grok AI chatbot account on X being used to create and share undressed images of people — which may amount to intimate image abuse or pornography — and sexualised images of children that may amount to child sexual abuse material,” Ofcom said in a statement.
- Advertisement -
Ofcom said it contacted X on Monday, 5 January, setting a deadline of Friday, 9 January, to explain what steps it has taken to protect UK users. The company responded by the deadline, and the regulator said it conducted an expedited assessment of the available evidence as a matter of urgency.
The case centers on AI-generated deepfake images and the adequacy of X’s safeguards. In response to public outcry, xAI — the Elon Musk-led firm behind Grok — said it would restrict image generation and editing to paying subscribers as it addressed lapses that allowed users to create sexualised content of others, often without consent.
Critics across government and civil society described the paywall move as insufficient. Ireland’s Minister of State with responsibility for AI, Niamh Smyth, called the change “window dressing,” saying stronger protections are needed. The Children’s Ombudsman, Dr. Niall Muldoon, said the update “makes no major difference,” adding: “What you’re saying is you’ve got an opportunity to abuse, but you have to pay for it.”
Separately, Ireland’s media regulator, Coimisiún na Meán, said it is engaging with the European Commission over the concerns raised by Grok’s image tools and potential harms. Smyth said X has contacted her office and representatives from the company will meet with her in the coming weeks following her earlier request.
The international backlash has gathered pace. Malaysia temporarily blocked access to Grok, joining a growing list of countries acting after allegations of nonconsensual manipulated images and content involving minors. Indonesia last week became the first country to temporarily deny access to the bot. Malaysia’s communications regulator said the restriction follows repeated misuse “to generate obscene, sexually explicit, indecent, grossly offensive, and non-consensual manipulated images, including content involving women and minors.”
In an email response to Reuters seeking comment, xAI replied with what appeared to be an automated message: “Legacy Media Lies.” X did not immediately respond to a request for comment.
The UK’s Online Safety Act imposes duties on platforms to remove illegal content, protect users from harm — particularly children — and to have robust systems preventing the dissemination of child sexual abuse material and intimate image abuse. If Ofcom determines X has fallen short, it can require specific remedial steps and, if those are not taken, impose significant fines.
The investigation places fresh scrutiny on the risks of generative AI when embedded in large social platforms, where the speed and scale of distribution can magnify abuse. It also raises questions about whether paywalling powerful tools meaningfully curbs misuse or simply shifts access to those willing to pay.
Ofcom did not set out a timetable for next steps, but said it moved rapidly in this case given potential harm. The outcome will signal how the UK intends to enforce its online safety rules in the age of AI — and whether platforms like X can demonstrate that their safety systems keep pace with fast-evolving generative technologies.
Additional reporting: Reuters
By Abdiwahab Ahmed
Axadle Times international–Monitoring.
