The AI-powered image generator, Grok, developed by Elon Musk's company xAI, has been found to create thousands of non-consensual images of women in "undressed" and "bikini" photos. The tool, which is part of the X social media platform, has been generating these images in response to user prompts, often resulting in women being digitally stripped of their clothes.
While not containing explicit nudity, these images are deemed a serious abuse instance, with some experts labeling it as the most widespread mainstream use of non-consensual AI-generated intimate imagery. Critics argue that X's failure to adequately address this issue has made it easier for users to create such content and normalize its creation.
The use of Grok for creating sexualized images is part of an increasingly concerning trend in which explicit deepfakes have become more advanced and accessible, with services generating at least $36 million each year. The National Center for Missing and Exploited Children has reported a 1,325% increase in reports involving generative AI between 2023 and 2024.
Several countries are now taking action against the misuse of such AI technology, including Australia's online safety regulator targeting one of the biggest nudifying services with enforcement action. Meanwhile, officials in France, India, and Malaysia have raised concerns or threatened to investigate X over its role in creating non-consensual imagery.
The issue has sparked calls for greater regulation and action from lawmakers and regulators. The TAKE IT DOWN Act, passed by Congress last year, makes it illegal to publicly post non-consensual intimate imagery, including deepfakes. Online platforms will soon be required to provide a way for users to flag such content, which must be responded to within 48 hours.
While not containing explicit nudity, these images are deemed a serious abuse instance, with some experts labeling it as the most widespread mainstream use of non-consensual AI-generated intimate imagery. Critics argue that X's failure to adequately address this issue has made it easier for users to create such content and normalize its creation.
The use of Grok for creating sexualized images is part of an increasingly concerning trend in which explicit deepfakes have become more advanced and accessible, with services generating at least $36 million each year. The National Center for Missing and Exploited Children has reported a 1,325% increase in reports involving generative AI between 2023 and 2024.
Several countries are now taking action against the misuse of such AI technology, including Australia's online safety regulator targeting one of the biggest nudifying services with enforcement action. Meanwhile, officials in France, India, and Malaysia have raised concerns or threatened to investigate X over its role in creating non-consensual imagery.
The issue has sparked calls for greater regulation and action from lawmakers and regulators. The TAKE IT DOWN Act, passed by Congress last year, makes it illegal to publicly post non-consensual intimate imagery, including deepfakes. Online platforms will soon be required to provide a way for users to flag such content, which must be responded to within 48 hours.
this is getting super scary like what's next AI making our deepest fears come true? 
and the fact that it's being used to create these non-consensual images of women is just disgusting 
I don't think X should be allowed to get away with not doing more to stop this and regulate it ASAP 
. But on the other hand, I'm all for exploring new tech that can help us solve some of our biggest problems, like increasing cyber-safety 
.
. If we're gonna create new tech to combat this issue, we need to make sure we're also educating users about digital responsibility and healthy online interactions 
.
. We need more collaboration between governments, tech companies, and civil society organizations to develop effective solutions 
.
. And the whole situation with deepfakes is just wild 
... I'm all for innovation, but not at the cost of our personal safety and dignity 
... We need stricter regulations, ASAP 
...
I don't care if it affects businesses or whatever... our safety online has to be a priority 
And yeah, it's not just Grok - all those deepfake services out there are making this problem way worse 
. Meanwhile, our lawmakers are still playing catch-up and trying to figure out how to regulate this stuff.
. The TAKE IT DOWN Act is a step in the right direction, but it's not enough - we need stricter laws that hold these tech companies accountable for their actions 
. And what about the users who are being exploited by these services? Don't they deserve some protection too? 
.
The real question is, what's next? Are we gonna see more of these AI-generated images being created and shared on social media platforms? Or are our lawmakers gonna step up and take action to prevent this kind of exploitation? 
.
. And the lack of action from X on this issue is concerning 
. We need more transparency and accountability from these platforms, like who's reviewing user prompts and what measures are being taken to stop this from happening again 
. It's about time we take steps to regulate the tech industry and protect our online rights 
.
. the fact that they're being generated by a tool designed for creative purposes is just messed up 
. what's next, AI-generated revenge porn or something? 
. it's not just about the tech itself, it's about the people who use it with malicious intent 
.
. It's like, what's next? Is this going to become the norm or are we actually going to do something about it? 
. Can't we find a way to make AI work for good instead of evil? 
Elon Musk needs to do better than just developing a tool that can be misused in such a way 
we need stricter regulations on AI tech like this, especially if it's being used to create explicit content that can be super damaging 
I'm so tired of seeing women getting exploited online and it's time for action 
some ppl r saying that platforms like X should have more control over what gets posted online so these kinda pics don't spread 
what's next? will we see AI-generated vids of ppl too? 
.
It's not like they're just going to ignore the issue and hope it goes away. The fact that Australia is taking action against one of these services is a good start, but we need more than that 
.
. I'm not saying we should just throw our hands up in the air and give up, but we do need to take a step back and think about the consequences of our actions 
this is so not cool 
. Can't we just have some basic rules of decency online? 