Smartphone app checks vital signs with face scan

Every morning before another long shift, employees at a Kajima construction site in Singapore must have their vital signs checked by an artificial intelligence-driven app that has been described by the company as its ''first line of defense'' against an outbreak. Francesca Lynagh reports.

Video Transcript

- Every morning before their shifts as a Kajima construction site, workers check their vital signs. All it takes is a 45-second scan of their faces using an app developed by Singapore start up Nervotec. The app takes note of heart rate, oxygen levels, respiration rate, and even stress levels using artificial intelligence-based technology.

So how does it work? The Nervotec app uses remote photoplethysmography and AI to capture and analyze the user's vitals. The smartphone camera measures the differences in the reflectivity of light that hits the user's skin, which corresponds to the different pulse rates of the body. Computer vision and predictive analysis AI then monitor the user's face and conclude the readings for their vital signs. Here's Nervotec Founder Jonathan Lau.

JONATHAN LAU: What we do is, we use the white light that's now reflecting off my face, we apply smart computer vision techniques to first identify the face, then filter this white light into the channels that we are interested in, and then deriving the vital signs from those channels.

- Kajima has been using Nervotec's app at its work sites in Singapore since December 2020, to complement daily temperature screenings. It's part of a government-initiated program which provides companies with technology still in their trial stages to help them adjust to the new norms. Kajima senior manager Tan Kee Chuan says the Nervotec app is his company's first line of defense against another health crisis.

TAN KEE CHUAN: And the application acts as a first line of defense by scanning the workers just by using the hand-phone. It is very convenient provided that the worker adhere today to this scanning on a daily basis. So we do have our own temperature monitoring system installed as a second line of defense, to reject all the personnel who are deemed unfit to work.

- Similar apps that utilize smartphone cameras to scan users vital signs do exist, but Nervotec claims that its technology goes one step further by using the data to offer a diagnosis of the user's health condition. Professor Chwee Teck Lim is the director of the National University of Singapore Institute of Health Innovation and Technology.

CHWEE TECK LIM: So what Nervotec is proposing is could be potentially a game changer. They are trying to use the smartphone camera couple with an AI-driven app to capture image of the face, and then they have to be able to measure the vital signs. So currently, I think they claim that they can obtain accuracy of down to two beats per minute for heart rate, and also two percentage in terms of oxygen saturation. But it remains to be seen, I think we still have to go through this FDA regulatory testing before we can determine how accurate this technology is. On our system and SPOT is on point.

- App is still under review but Lau said, there is a significant interest in the technology.

CHWEE TECK LIM: We see the most traction coming from health care providers, both private and public. More often than the authorities, because the ability to use rPPG and to have constant remote patient monitoring, without the need for additional manpower or equipment, is really a big problem solver for a lot of health care providers globally.