For the blind and visually impaired, everyday tasks such as taking out the trash or walking the dog can sometimes become quite difficult. Having simple conversations could also become a challenge. While they can hear what the other person is saying, emotional cues like facial expressions often go undetected. One company is looking to use artificial intelligence to help the visually impaired “see” facial expressions.

Facing Emotions App Turns Emotions Into Sounds

Chinese electronics company Huawei’s new app, Facing Emotions, identifies the 7 basic human emotions of irritation, contempt, sorrow, fear, anger, surprise and happiness. The app then turns those emotions into unique sounds to help the visually impaired learn how the person on the other side of the conversation is feeling.

Huawei partnered with the Polish Blind Association to help develop and test out the app. Visually impaired members were integral in shaping the app giving input on things like sound and color. The Facing Emotions app is also helping to launch the Huawei Mate 20 Pro which features an advanced camera, artificial chip, and a powerful processor.

App Scans Position of Facial Features to Interpret Emotion

The app uses the Mate 20 Pro’s advanced camera to the scan the position of facial features such as nose, eyes, brows, and mouth. An algorithm then interprets that information and translates the emotion into a corresponding sound which the user hears through the device’s headphones. The app does not require an internet connection to work.

The sounds were created by blind composer Tomasz Bilecki. The phone can also be used with a 3D printed holder that allows visually impaired people with canes to use the app hands-free. The unique holder was developed by designer Janek Kochanski.

“The Facing Emotions campaign embodies Huawei’s mission to bring new technology to everyone, especially those that need it most,” said Wang Tao, president of Huawei’s Consumer Business Group.

Check out our articles on Gait recognition identifying people by body shape and AI helping to conserve wildlife.