Big tech gone bad: Google bans accounts based on fake child exploitation filter

The application of artificial intelligence in various areas of life is mostly positive, but what happens if machine learning algorithms can’t tell the difference between a clean photo of health and child exploitation?

The New York Times reported today that San Francisco-based tech employee “Mark” — who works in content filtering for an oddly unnamed tech company — has been banned by Google LLC for “violating Google policies and potentially illegal.” After the child is sick and the child’s photos are taken. In this case, the child had problems with his penis and after being asked by a medical professional, the owner took pictures and sent them online through his Google account.

Google’s AI and filters have flagged the photos as child abuse without taking the situation into context. An incorrect or out-of-context Google block isn’t very surprising. But what happened next is arguably the most terrifying.

Google said in the report that the man, identified as Mark, was making child pornography and that a police investigation had begun. Fortunately, the San Francisco Police Department got the hang of it, but almost two years later, Google didn’t, and Mark is still locked out of his account.

Google’s only comment in response to the New York Times story was that “child sexual abuse is abhorrent and we are committed to preventing it from spreading across platforms.” No one can argue with this statement, but in this case, the photos were not offensive, but rather a legitimate health issue.

The case shows how dependent billions of people are on tech companies, and how a simple false positive can turn into something worse. In Markos’s case, in addition to losing his email account, Google Photos account and contact information, he also had a phone number blocked by Google Fi. The Google Fi phone number is linked to different accounts, which means it also loses two-step verification.

“The more eggs you have in one basket, the more likely the basket will break,” Mark told The Times.

Mark’s case is not unique. The Times also cited a similar case from a Texas father in which Google posted medical-related photos online as offensive. In the Texas case, as in Mark’s case, Google suspended the father’s account and the Robot Technology staff in Mountain View denied any appeal.

Google and other big tech companies should actively seek out and filter offensive content. However, Google’s inability to find human staff to look into the appeal and sort out the situation does not reflect positively on Google and other tech companies.

Fortunately, in both Mark’s and Texas’ cases, the medical issue with their children was resolved, but there is no medical guidance for dealing with Google and other tech companies who make up their minds and falsely label someone as a child. Pornography.

Photo: Paradise Pancake/Wikimedia Commons