Although there are a lot of weird headphones and TWS earbuds floating around the market, I bet none of those can track your facial expressions. I mean they are made for users to listen to AC/DC or System of a Down, not to track cheek muscles, right? Well, that is not the perception of the researchers at the Cornell University of New York who developed a pair of headphones to track facial expressions in real-time.
Now, the C-Face comes equipped with two tiny RGB cameras attached to each of its earcups. Thanks to these two cameras, the device is able to record the changes in facial contours at the time of facial muscle movements.
So, after the device captures the changes in the facial muscles, the images are re-built using a deep learning model and computer vision. And as the resulting images are in 2D, the researchers deployed an AI model, that specializes in classifying, detecting, retrieving images, to reconstruct the contour expressions.
Possible Applications
Now, according to the researchers, the facial expressions, determined by 42 distinct feature points, can be turned into various emojis, including natural, kiss-face, and angry-face. Moreover, the tech can also enable “silent speech commands” that could help users control a music player just with facial expressions.
