Intelligent and unobtrusive new camera technology has the potential to prolong life by highlighting early warning signs associated with health conditions.

Debuted at CES

Tragic and disruptive as it has been, the pandemic continues to give way to innovation in a wide range of sectors, not least, of course, healthcare. With its newly developed HealthCam, Mitsubishi Electric is set to make its debut in the market and introduce an entirely new way to help detect illness and even prevent potentially fatal emergency situations.

HealthCam, unveiled to the world at CES 2022, is a piece of computer vision technology that combines facial and thermal detection to monitor vital signs including heart rate and body temperature. The technology is highly adaptable and while it is still in development, Mitsubishi Electric also sees it being able to detect accidents such as choking, the risk of syncope, and other emergencies.

Two cameras mounted side-by-side are calibrated to share information, which comes from monitoring subtle changes in color, temperature and shape imperceptible to an observer. While the system records body temperature as a direct measurement, facial color and corporeal shape variations require highly sensitive detection and inference capabilities.

Unobtrusive Analysis

In HealthCam’s current form, the cameras are integrated into a smart mirror. Alan Sullivan, Manager of the Computer Vision Group at Mitsubishi Electric Research Labs who is leading the project, cites the unobtrusiveness of this setup as an advantage that makes it easy to deploy the system in people’s homes as well as in locations such as elderly care facilities, and pharmacies that offer health-testing. With this equipment, a person’s condition can be analyzed multiple times a day with no effort required on their part.

Sullivan is clear that Covid-19 was a motivating factor for the technology’s development. The ability to detect symptoms early could have helped prevent the spread of the virus and many deaths, he notes. The technology is not yet able to gauge blood oxygenation, a key Covid indicator, but that aspect is in development. However, Sullivan says early detection of other ailments, or of instances such as choking or even stroke, can help prevent deaths among the elderly, particularly in retirement homes.

In the later stages of life, "so many things can turn into a disaster if they’re not caught early", he says. A stroke, for instance, is not always apparent to the sufferer. Diseases can also show themselves in subtle ways, Sullivan points out.

Wearable technology can be useful as a health monitoring tool, but only up to a point, since the user must remember to put it on and monitor it regularly, which in some cases is easier said than done. A watch, wristband or ring is also not always a desirable accessory, especially for the elderly. Here again, the essentially invisible nature of HealthCam comes into play, enabling a contactless, comprehensive and intelligent solution to look out for illness.

From the Driver’s Seat to the Wider World

HealthCam is the result of an internal pitch across the company that called for ideas that use or combine existing technologies for new applications with a focus on people-centric solutions – products aimed at improving people’s lives.

It began as a device to monitor the heart rate of drivers and pre-empt accidents caused by drowsiness or cardiac arrest. Having established the ability to measure heart rate, company researchers began to explore further possible applications. Mitsubishi Electric was a pioneer in developing AI face-detection technology, which was already operational. This led to research projects around face landmark detection in conjunction with heart-rate monitoring, and the development of a user interface.

By effectively harnessing years of research collaboration between labs in the US and Japan for driver monitoring, the Mitsubishi Electric Innovation Center (MELIC) was able to quickly identify and target new and untapped business opportunities in the healthcare domain.

The process offered a number of staff an unprecedented opportunity to create a new product with serious commercial potential and set a new pace for development: HealthCam rapidly moved from a concept to a working product within the space of a year. Still, there is work to be done. Sullivan says Mitsubishi Electric is refining the technology and making it more robust. One of the challenges has been ensuring facial reading accuracy of mask-wearers, but early results look promising.

A Platform for Self-awareness

HealthCam is designed to operate in the cloud rather than following an edge computing model, which is more common for solutions of its kind. Sullivan sees this reducing both complexity and cost. To ensure privacy, only processed data from the captured images, not the images themselves, will ever leave the device to undergo processing in the cloud environment.

Depending on its application, the product may or may not require regulatory approval. In hospital settings, it is mandatory; outside, the requirements change. Sullivan says pharmacies and elderly care facilities are obvious prospects with high potential. Gaming and streaming are areas where it could also have an impact by acting as a medium for advice to improve performance or prevent over-exertion. In the end, it will be unlikely to operate everywhere at once, and Mitsubishi Electric is still in the process of narrowing the field.

Sullivan describes HealthCam as a platform rather than a product – one that can assimilate new technologies as they take form. He sees breath analysis and vision analysis as realistic future possibilities. The former, for instance, could help identify the presence of diabetes.

Of course, HealthCam does not claim or seek to be a doctor in itself. By plugging experts into the platform, it will be able to operate as a source of information and insight with data analysis at its core. "Our main goal is early warning," Sullivan explains. "Then professional physicians can kick in and help people avoid long-term harm. This isn’t designed to decide whether people are healthy or not, but to lead them to visit a medical professional."

(This article was originally published on April 27, 2022)

The content is true and accurate as of the time of publication.Information related to products and services included in this article may differ by country or region.