New Law Allows Experts to Test AI Tools for Child Abuse Images Before They're Released
A major overhaul is being made to UK child safety agencies' ability to detect potentially problematic artificial intelligence tools. This change is aimed at tackling the alarming rise in AI-generated child sexual abuse material (CSAM). According to a recent report, cases of such images have more than doubled from 199 instances in 2024 to 426 in 2025.
Under the new legislation, tech companies and designated child safety organizations will be granted permission to evaluate AI models used by chatbots like ChatGPT and image generators like Google's Veo 3. The aim is to ensure these tools are designed with robust safeguards that prevent them from creating child abuse images.
Experts involved in this process hope it will help identify potential risks early, allowing developers to address issues before they reach the public. However, critics argue that AI-generated CSAM poses a significant threat to children's safety online and offline.
The Internet Watch Foundation reported that 94% of recent AI-generated images targeted girls, with newborns to two-year-olds increasingly being depicted in these images. Survivors have also been victimized through AI tools, which can create sophisticated, photorealistic child abuse material at the click of a button.
Childline has seen an increase in counseling sessions where AI is mentioned as a concern. The helpline reported 367 sessions between April and September this year, four times more than during the same period last year. Many children are being bullied online using AI-generated content or blackmailed with AI-faked images.
The government believes that by testing AI tools before they're released, it can help prevent the creation of child abuse images at source. The changes to the crime and policing bill also introduce a ban on possessing, creating, or distributing AI models designed to generate CSAM.
While the law change holds promise, many experts and organizations are emphasizing its importance in preventing further harm to children.
A major overhaul is being made to UK child safety agencies' ability to detect potentially problematic artificial intelligence tools. This change is aimed at tackling the alarming rise in AI-generated child sexual abuse material (CSAM). According to a recent report, cases of such images have more than doubled from 199 instances in 2024 to 426 in 2025.
Under the new legislation, tech companies and designated child safety organizations will be granted permission to evaluate AI models used by chatbots like ChatGPT and image generators like Google's Veo 3. The aim is to ensure these tools are designed with robust safeguards that prevent them from creating child abuse images.
Experts involved in this process hope it will help identify potential risks early, allowing developers to address issues before they reach the public. However, critics argue that AI-generated CSAM poses a significant threat to children's safety online and offline.
The Internet Watch Foundation reported that 94% of recent AI-generated images targeted girls, with newborns to two-year-olds increasingly being depicted in these images. Survivors have also been victimized through AI tools, which can create sophisticated, photorealistic child abuse material at the click of a button.
Childline has seen an increase in counseling sessions where AI is mentioned as a concern. The helpline reported 367 sessions between April and September this year, four times more than during the same period last year. Many children are being bullied online using AI-generated content or blackmailed with AI-faked images.
The government believes that by testing AI tools before they're released, it can help prevent the creation of child abuse images at source. The changes to the crime and policing bill also introduce a ban on possessing, creating, or distributing AI models designed to generate CSAM.
While the law change holds promise, many experts and organizations are emphasizing its importance in preventing further harm to children.
... We gotta ask ourselves, what's the value of a technology if it can be used to hurt so many innocent lives? The fact that 94% of these images target girls & newborns highlights how vulnerable we've made our children online. 
... We need to keep pushing for better safeguards, not just technical solutions. It's time for us all to think about the human cost of AI & ensure we're building technologies that serve humanity, not harm it 
, I don't know how tech companies can afford not testing their AI tools for child abuse images... Like, it's not that hard to do a quick scan, right? 
And what's with the lack of data on AI-generated CSAM? We need more research done ASAP! 
. And have you seen those photorealistic child abuse material at the click of a button? 
It's like something out of a horror movie! 
I'm glad they're taking this seriously, tbh 
. It's crazy how fast AI-generated CSAM is spreading online, especially with girls and newborns being targeted 
. The fact that 94% of these images are targeting young girls is just heartbreaking 
. We need to make sure these tech companies are held accountable for creating tools that can prevent this kind of abuse 
. And I think banning AI models designed to generate CSAM is a good start, but we need to make sure these tools are being tested properly too 
. The child safety agencies should be able to identify potential risks early on, and then developers can address those issues before they reach the public 
.
Let's just shut this whole thing down ASAP 
.
i mean think about it if these AI tools can create super realistic images of kids just imagine how messed up that is 
i feel bad for the kids who are being victimized online and offline 
and i think this law change is a good start but it's not like it's gonna magically solve everything 
we need to keep pushing forward and make sure these AI tools are super safe 
... but also kinda worrying that tech companies are being allowed to test them before release... what if it's not good enough? 
...
hope this law change helps reduce the numbers, it's getting too out of hand for my blood 
IT'S ABSOLUTELY DISGUSTING THAT PEOPLE CAN CREATE SUCH DEPRAVING CONTENT WITH JUST A CLICK OF A BUTTON! 
That'd be a step in the right direction imo.