(Credit: Kristian Buus/In Pictures via Getty Images)
I've built my career on helping people solve tech problems, but when it comes to something like xAI's Grok generating child sexual abuse material (CSAM) and non-consensual intimate images, I shouldn't have to offer advice on how to protect yourself.
Last month, the Grok AI chatbot on X temporarily allowed people to request that it transform clothed photos of real women (and in some cases, children) into AI-generated images of them wearing bikinis or lingerie and post it publicly via @Grok. Amid pushback, X owner Elon Musk argued that "Anyone using Grok to make illegal content will suffer the same consequences as if they upload illegal content." But he was reportedly hesitant to axe the feature, citing "over censorship," until regulators threatened legal action.
His focus on “illegal” content sidesteps the underlying issue: Platforms are never value neutral. They encourage the behavior that their creators want to see. Or, to put it another way, the purpose of a system is what it does. X's actions have shown that the preferences of people who commit digital sexual harassment are a higher priority than those being harassed.
What the System Does
First, let’s clear one thing up: while X says it is taking measures to prevent Grok from generating CSAM and non-consensual sexual images, that doesn’t exactly line up with reality. While public image editing via the Grok X account is now limited to paid subscribers, edit buttons are still on every photo posted to X. Anyone who clicks (paid or free) is simply redirected to the Grok app or Grok.com. Grok's replies then appear in a private conversation rather than publicly.
(Credit: PCMag / X)Furthermore, while some specific prompts with obvious ill-intent—such as the commonly used “put her in a bikini”—will return a generic “Content Moderated” response, it’s still trivially easy to generate revealing or vulgar images by using less direct descriptions. The only difference now is there’s no central public feed of the images Grok generates.
X defenders might argue that other chatbots can be manipulated to create non-consensual imagery. Google's Gemini, for example, refuses to generate sexualized images of real people, but it can be coerced into changing a person’s outfit with the right creative prompt. However, in my testing, it didn't create the same kind of revealing imagery that Grok readily serves up.
This misses the more important point that Grok’s image-editing tool is built directly into every picture on the site, with no way to opt out, something no other social media site has embraced.
The UX Design of Sexual Harassment
It might be tempting to argue that this is picking nits. After all, Photoshop has been around forever, right? Crass, low-life internet users have been editing women’s faces onto nude bodies since the first computers came online. Is it really so different if Grok makes it a two-click affair?
If this thought crossed your mind, I’d encourage you to find the nearest UX designer and ask them how much better a return they get when the shopping experience takes as few clicks as possible. In any other context, we intuitively understand that the easier it is to do something, the more often it happens. It’s why Amazon has "Buy Now" buttons to speed up the checkout process, and why canceling your subscription is always harder than signing up.
When a social media platform like X gives its users a button to easily edit any image posted to the feed and pairs it with insufficient guardrails, it's designing a system that encourages abuse. The consent of the users posting or depicted in the images is secondary to the desires of the users manipulating the photos, if their consent is considered at all. These choices telegraph which people and values are important to a platform’s creators. Elon Musk claims that he values free speech, but the freedom to sexually harass people should not be respected.
'She Was Asking For It'
My instincts as a service journalist mean I want to provide some kind of practical advice for the people who are worried about this kind of harassment. We already have several guides on managing X’s toxicity, opting out of training Grok’s model, or straight up deleting your account.
But all that advice feels like the digital equivalent of telling women not to wear short skirts or go out at night. The problem is not with the people posting pictures of themselves or their children. It’s with the users who believe that it is acceptable to turn their images into revealing or degrading photos against their will and the platforms that empower them.
Every social media company chooses who they want on their platforms, and they can make those platforms safe and provide tools that prevent harassment, or they can prioritize the freedom of users who delight in causing harm to others. There's a reason you don’t see non-consensual nudes under every Instagram post or TikTok video. Or that people who want to use generative AI to make CSAM typically have to hide in private Telegram channels.
This issue is not a question of technology; it's a question of values. How should our digital spaces be structured, and who should get to feel safe in public? Framed that way, the answer is painfully obvious. You should be able to exist online without being sexually harassed by strangers. And people who use AI to create non-consensual sexualized images—especially of children—should not feel protected by the tools and platforms they use.