Report urges fixes to online child exploitation CyberTipline before AI makes it worse

The service was established by Congress as the main line of defense for children who are exploited online. By law, tech companies — must report any child sexual abuse material they find on their platforms to the system, which is operated by the National Center for Missing and Exploited Children. After it receives the reports, NCMEC attempts to find the people who sent or received the material — as well as the victims, if possible. These reports are then sent to law enforcement.

While the sheer amount of CyberTipline reports is overwhelming law enforcement, researchers say volume is just one of several problems core to the system. For instance, many of the reports sent by tech companies lack important details, such as enough information about an offender’s identity, the report said. This makes it hard for law enforcement to know which reports to prioritize.

“There are significant issues with the entire system right now and those cracks are going to become chasms in a world in which AI is generating brand-new CSAM,” said Alex Stamos, using the initials for child sexual abuse materials. Stamos is a Stanford lecturer and cybersecurity expert.

The system is behind technologically and plagued by a constant challenge among government and nonprofit tech platforms: the lack of highly skilled engineers, who can get paid far higher salaries in the tech industry. Sometimes those employees are even poached by the same companies that send in the reports.

Then there are legal constraints. According to the report, court decisions have led the staff at NCMEC to stop vetting some files (for instance, if they are not publicly available) before sending them to law enforcement. Many law enforcement officials believe they need a search warrant to access such images, slowing down the process. At times, multiple warrants or subpoenas are needed to identify the same offender.

It’s also easy for the system to get distracted. The report reveals that NCMEC recently hit a milestone of a million reports in a single day due to a meme that was spreading on multiple platforms — which some people thought was funny and others were sharing out of outrage.

“That day actually led them to make some changes,” Stamos said. “It took them weeks to get through that backlog” by making it easier to cluster those images together.

The CyberTipline received more than 36 million reports in 2023, nearly all from online platforms. Facebook, Instagram, and Google sent in the highest number of reports. The overall number has been dramatically increasing. The reports come from big companies like Google, Amazon, and Meta as well as smaller ones.

Nearly half of the tips sent last year were actionable, meaning NCMEC and law enforcement could follow up.

Hundreds of reports concerned the same offender, and many included multiple images or videos. Around 92% of the reports filed in 2023 involved countries outside the U.S., a large shift from 2008 when the majority involved victims or offenders inside the U.S.

Some are false alarms. “It drives law enforcement nuts when they get these reports that they perceive are definitely adults,” Grossman told reporters. “But the system incentivizes platforms to be very conservative or to report potentially borderline content, because if it’s found to have been CSAM and they knew about it and they didn’t report it, they could receive fines.”

One relatively easy fix proposed in the report would improve how tech platforms label what they are reporting to distinguish between widely shared memes and something that deserves closer investigation.

The Stanford researchers interviewed 66 people involved with CyberTipLine, ranging from law enforcement to NCMEC staff to online platform employees.

The NCMEC said it looked forward to “exploring the recommendations internally and with key stakeholders.”

“Over the years, the complexity of reports and the severity of the crimes against children continue to evolve. Therefore, leveraging emerging technological solutions into the entire CyberTipline process leads to more children being safeguarded and offenders being held accountable,” it said in a statement.

Among the report’s other findings:

— The CyberTipline reporting form doesn’t have a dedicated field for submitting chat-related material, such as sextortion messaging. The FBI recently warned of a “huge increase” in sextortion cases targeting children — including financial sextortion, where someone threatens to release compromising images unless the victim pays.

— Police detectives told Stanford researchers they are having a hard time persuading their higher-ups to prioritize these crimes even after they present them with detailed written descriptions to emphasize their gravity. “They wince when they read it and they don’t really want to think about this,” Grossman said.

— Many law enforcement officials said they were not able to fully investigate all reports due to time and resource constraints. A single detective may be responsible for 2,000 reports a year.

— Outside the U.S., especially in poorer countries, the challenges around child exploitation reports are especially severe. Law enforcement agencies might not have reliable internet connections, “decent computers” or even gas for cars to execute search warrants.

— Pending legislation passed by the U.S. Senate in December would require online platforms to report child sex trafficking and online enticement to the CyberTipline and give law enforcement more time to investigate child sexual exploitation. Currently, the tipline doesn’t offer straightforward ways to report suspected sex trafficking.

While some advocates have proposed more intrusive surveillance laws to catch abusers, Stamos, the former chief security officer at Facebook and Yahoo, said they should try simpler fixes first.

“There’s no need to violate the privacy of users if you want to put more pedophiles in jail. They’re sitting right there,” Stamos said. “The system does not work very well at taking the information that currently exists and then turning it into prosecutions.”