Researchers trick facial recognition systems

McAfee recently undertook research to ascertain whether computer-generated images that look like one person would get classified as another person. Jai Vijayan from Dark Reading reports on the findings.

Neural networks powered by recent advances in artificial intelligence and machine learning technologies have become increasingly adept at generating photo-realistic images of human faces completely from scratch.

The systems typically use a dataset comprised of millions of images of real people to ‘learn’ over a period of time how to autonomously generate original images of their own.

At the Black Hat USA 2020 virtual event recently, researchers from McAfee showed how they were able to use such technologies to successfully trick a facial-recognition system into misclassifying one individual as an entirely different person. As an example, the researchers showed how an individual on a no-fly list could trick a facial-recognition system used for passport verification into identifying him as another person.

“The basic goal here was to determine if we could create a fake image, using machine learning models, which looked like one person to the human eye, but simultaneously classified as another person to a facial recognition system,” says Steve Povolny, Head of Advanced Threat Research at McAfee.

To do that, the researchers built a machine-learning model and fed it training data: a set of 1,500 photos of two separate individuals. The images were captured from live video and sought to accurately represent valid passport photos of the two people.

The model then continuously created and tested fake images of the two individuals by blending the facial features of both subjects. Over hundreds of training loops, the machine-learning model eventually got to a point where it was generating images that looked like a valid passport photo of one of the individuals: even as the facial recognition system identified the photo as the other person.

Povolny says the passport-verification system attack scenario — though not the primary focus of the research — is theoretically possible to carry out. Because digital passport photos are now accepted, an attacker can produce a fake image of an accomplice, submit a passport application, and have the image saved in the passport database. So if a live photo of the attacker later gets taken at an airport — at an automated passport-verification kiosk, for instance — the image would be identified as that of the accomplice.

“This does not require the attacker to have any access at all to the passport system; simply that the passport-system database contains the photo of the accomplice submitted when they apply for the passport,” he says.

The passport system simply relies on determining if two faces match or do not match. All it does is verify if a photo of one person is identified against a saved photo in the back end. So such an attack is entirely feasible, though it requires some effort to pull off, Povolny says.

“It is less likely that a physical passport photo that was mailed in, scanned, and uploaded to this database, would work for the attack,” he notes.

Generative Adversarial Networks

McAfee’s research involved the use of a so-called Generative Adversarial Network (GAN) known as CycleGAN. GANs are neural networks capable of independently creating data that is very similar to data that is input into them. For example, a GAN can use a set of real images of human faces or of horses to autonomously generate completely synthetic — but very real-looking — images of human faces and horses. GANs use what are known as generative networks to generate the synthetic data, and discriminative networks to continuously assess the quality of the generated content until it reaches acceptable quality.

CycleGAN itself, according to McAfee, is a GAN for image-to-image translation: translating an image of zebras to an image of horses, for example. One feature of the GAN is that it uses significant features of an image for translation, such as eye placement, shape of head, body size, and other attributes.

In addition to CycleGAN, the McAfee researchers also used a facial-recognition architecture called FaceNet, originally developed by Google for image classification. Building and training the machine-learning model took a period of several months.

“While we would have loved to have access to a real-world target system to replicate this, we are thrilled with the results of achieving positive misclassifications in white box and grey-box scenarios,” Povolny says.

Given the increasingly important role that facial recognition systems have begun playing in law enforcement and other areas, more proactive research is needed to understand all of the ways such systems can be attacked, he says.

“Anomaly testing, adversarial input, and more diverse training data are among the ways that vendors can improve facial recognition systems,” Povolny notes. “Additionally, defense-in-depth, leveraging a second system, whether human or machine, can provide a much higher bar to exploitation than a single point of failure.”

This story first appeared on Dark Reading. Part of the Informa Network, Dark Reading is a trusted online community for cyber security professionals, including CISOs, cyber security researchers and technology specialists. Covering the latest threats, vulnerabilities and cyber attacks, Dark Reading supports community members in keeping up with the latest in the sector.

Free Download: The Video Surveillance Report 2023

Discover the latest developments in the rapidly-evolving video surveillance sector by downloading the 2023 Video Surveillance Report. Over 500 responses to our survey, which come from integrators to consultants and heads of security, inform our analysis of the latest trends including AI, the state of the video surveillance market, uptake of the cloud, and the wider economic and geopolitical events impacting the sector!

Download for FREE to discover top industry insight around the latest innovations in video surveillance systems.

Download The Video Surveillance Report