A reporter tried the AI Instagram wants to use to verify age. Here's what it found

(CNN Business) Instagram is testing new ways to verify its youngest users' ages, including by using artificial intelligence that analyzes a photo and estimates how old the user is.

Meta-owned Instagram said in a blog post on Thursday that AI is one of three new methods it's testing to verify users' ages on the photo-sharing site. Users will be required to use one of the options to verify their age if they edit their birth date on Instagram from under age 18 to over 18. Instagram is testing these options first with its users in the United States. It already requires users to state their age when they start using the service, and employs AI in other ways to determine if users are kids or adults.

The move is part of an ongoing push to make sure the photo-sharing app's youngest users see content that is age-appropriate. It comes less than a year after disclosures from a Facebook whistleblower raised concerns about the platform's impact on younger users. Last year, Instagram came under fire when documents leaked by the whistleblower, Frances Haugen, showed it was aware of how the social media site can damage mental health and body image, particularly among teenage girls.

The technology comes from a London-based company called Yoti. An animated video that Instagram posted to its blog gives a sense for how Yoti's AI age-estimation works: A user is directed to take a video selfie on their smartphone (Yoti said this step serves as a way to make sure a real person is in the resulting image), and Instagram shares an image from that selfie with the company. Yoti's AI first detects that there is a face in the picture and then scrutinizes its facial features to determine the person's age.

Julie Dawson, Yoti's chief policy and regulatory officer, told CNN Business that its AI was trained with a dataset made up of images of people's faces along with the year and month that person was born. (Documentation the company released in May to explain its technology said it was trained on "millions of diverse facial images.")

"When a new face comes along, it does a pixel-level analysis of that face and then spits out a number — the age estimation with a confidence value," Dawson said. Once the estimation is completed, Yoti and Instagram delete the selfie video and the still image taken from it.

Verifying a user's age can be a vexing problem for tech companies, in part because plenty of users may not have a government-issued photo ID card that can be checked.

Karl Ricanek, a professor at the University of North Carolina Wilmington and director of the school's Face Aging Group Research Lab, thinks Yoti's technology is a good application of AI.

"It's a worthwhile endeavor to try and protect kids," he said.

Yet while such technology could be helpful to Instagram, a number of factors can make it tricky to accurately estimate age from a picture, Ricanek said, including puberty — which changes a person's facial structure — as well as skin tone and gender.

The recent documentation from Yoti indicates its technology is, on average, slightly less accurate at estimating the ages of kids who are between 13 to 17 and have darker skin tones than those with lighter skin tones. According to Yoti's data, its age estimate was off, on average, by 1.91 years for females ages 13 to 17 whose skin tones were categorized as the two darkest shades on the Fitzpatrick scale — a six-shade scale that's commonly used by tech companies to classify colors of skin — versus an average error of 1.41 years for females in the same age group whose skin tones were the two lightest shades on the scale. For kids between the ages of 13 to 17, the technology's estimate of how old they are was off by 1.56 years, on average, according to the document. (For teenagers overall, the average error rate is 1.52 years.)

What that means, in practice, is that there will be a lot of errors, said Luke Stark, an assistant professor at Western University in Ontario, Canada, who studies the ethical and social implications of AI. "We're still taking about a mean absolute error, either way, of a year to a year and a half," he said.

Several CNN employees — all adults over the age of 25 — tried an online demo of Yoti's age-estimation technology. The demo differs from the experience Instagram users will have in that it takes a selfie, rather than a short video, and the result is an age-range estimation, rather than a specific age estimation, Yoti's chief marketing officer, Chris Field, said.

The results varied: For a couple of reporters, the estimated age range was right on target, but for others it was off by many years. For instance, it estimated one editor was between the ages of 17 and 21, when they're actually in their mid-30s.

Among other issues, Stark is also concerned that the technology will contribute to so-called "surveillance creep."

"It's certainly problematic, because it conditions people to assume they're going to be surveilled and assessed," he said.