Face Masks Are Thwarting Facial Recognition Tech
New research from the National Institute of Standards and Technology (NIST) suggests face masks are hampering facial recognition systems.
Because the protective coverings obscure the bottom half of the face, they invalidate many of the results, leading to an error rate between five and 50 percent.
The margin of error seems to boil down to the shape of the face mask itself.
When you wear a face mask, you greatly reduce the chances of spreading any viral loads to nearby people if you cough, sneeze, or even talk. In the process, you might even protect yourself.
But there's an unintended consequence that comes from wearing face masks in the wake of COVID-19 (coronavirus): Your mask basically breaks facial recognition algorithms.
📲 Unlimited Pop Mech. Get best-in-class tech news and tips you can't find anywhere else, now.
In a preliminary study published this month, the National Institute of Standards and Technology (NIST) found that among 89 commercial facial recognition algorithms, the systems failed to identify people wearing digital face masks between five and 50 percent of the time, compared to images of the same people without a mask drawn on.
"With the arrival of the pandemic, we need to understand how face recognition technology deals with masked faces," study author Mei Ngan, a computer scientist at NIST, said in a prepared statement. "We have begun by focusing on how an algorithm developed before the pandemic might be affected by subjects wearing face masks."
Truthfully, it doesn't exactly take a computer scientist to invalidate an algorithm. Because they're just sets of instructions that a computer should follow to perform an operation—here, identifying a face—algorithms can go haywire when the input data doesn't align with the training data.
When a facial recognition algorithm is explicitly programmed to identify images of people by examining markers on the face—like the distance between the mouth and nose—it will inevitably fail more often when faced with images that obscure those same markers.
To test out that hypothesis, the NIST researchers set up "one-to-one" matching experiments, wherein the facial recognition algorithms must match one photo of a person with a different photo of the same person. This kind of verification is common in smartphones, which use one-to-one matching to see if your face matches the photo that you've stored for use with FaceID, for example.
Notably, this is not the kind of exercise that can identify an individual in a crowd from some sort of database. That kind of testing is called "one-to-many," and it's inherently more rigorous because the algorithm must match one image of a face to another image of that face, but among a dataset of many other faces, too.
NIST researchers set up the experiment so the 89 different facial recognition algorithms tried to come up with matches based on a sample of 6 million photos. The team digitally altered those images by applying various kinds of masks to the faces in the original images; because real-world masks are varied, the team created nine different examples that varied in shape, color, and nose coverage.
Then, the scientists checked to see how well the algorithms could match the faces in the original photos with the faces that were digitally altered to include a mask. They found a few general patterns:
👀 Across the board, algorithms struggle to identify faces when digital masks are applied. The most accurate algorithms failed to identify a non-mask wearer just 0.3 percent of the time, but when the researchers applied the masks, the best algorithms failed about 5 percent of the time, while others failed at higher rates between 20 and 50 percent.
👀 Algorithms are more likely to encounter a "failure to enroll or template" (FTE) error when a person is wearing a mask. Normally, facial recognition systems take measurements from around the face, like the distance between the eyes and nose, and compare them to measurements in another photo. FTE errors occur when an algorithm can't extract a person's facial features from an image, making it impossible to make a comparison to another photo.
👀 Less nose = less accuracy. The study looked at three levels of nose coverage in masks—low, medium, and high—and found that masks with more nose coverage lead to a higher incidence of error.
There are a few caveats: This study took digitally drawn masks into consideration—not actual face masks. Plus, the algorithms were built prior to the pandemic, so they weren't explicitly designed to deal with faces that are partially obscured by masks.
Researchers need to do more work to verify these results. Later this summer, NIST plans to test the accuracy of algorithms that companies intentionally developed with masked faces in mind. The team will also conduct one-to-many verification tests to see how well facial recognition systems can match up a masked individual to a database.
You Might Also Like
