Featured Science Technology

EmoSense, An AI-Powered System Detects Emotions Based On Physical Movements


A team of researchers at the Hefei University of Technology in China and various other universities in Japan have developed a unique emotion-sensing system that recognizes people’s emotions based on their body gestures. The new system EmoSense was presented in a paper published in arXiv.

One of the researchers Yantong Wang explains that body gestures contain rich mood expressions for emotion recognition. Interestingly, human body gestures also affect wireless signals via shadowing multi-path effects when antennas are used to detect behavior. These signals form unique patterns and fingerprints in the temporal-frequency domain for different gestures.

The team noted that human gestures can impact the wireless signals, generating characteristic patterns that could be used in emotion recognition. This inspired them to create a system that can identify these patterns and recognize emotions based on physical movements.

In the study, the team focused on tow aspects, first, they examined how body gestures can be detected by analyzing the characteristic patterns they leave on wireless signals. After that, they started to develop a system that could recognize emotions based on body gestures.

Most vision-based or sensor-based tools for emotions recognition are either uncomfortable to wear or rely on specific hardware, which affects their efficacy in real-time. The system developed by Wang and his team is less obstructive and works by analyzing wireless channel responses through data mining and detecting emotions based on gestures and wireless fingerprints.

The researchers tested EmoSense’s performance on 360 cases and compared them to several sensor and vision-based tools. The system performed similarly to the other approaches without the need for any expensive hardware and is easy to implement. Because it works by analyzing wireless signals, EmoSense also leads to lower privacy-related issues.

Future applications.

The system developed by Wang and his colleagues could have several interesting applications, for instance, it could be used during theatre rehearsals to determine whether a play gets the desired emotional reactions from the audience. The researchers plan to develop the system further and also explore other aspects of emotion recognition.

According to the team, a key limitation of the system is that it is data-driven. It relies primarily on quantitative observations and does not consider more complex psychological aspects of emotions.

“EmoSense and its rivals hinge upon human gestures as an expression of emotion, which remains a blur, so far,” Wang explained. “For example, dishonest people can deceive the system by intentionally behaving in certain ways. A possible solution to this problem is to leverage the multi-modality feature of emotion.”

As psychology knowledge is also very important for understanding human emotion, it might be more reasonable to couple both data and psychology knowledge to attain more reliable and accurate emotion recognition. Additionally, the physical expression of emotion is affected by many congenital and acquired factors, some of which are totally out of control. Therefore, it is important to clarify the potential scenarios before we deploy the system.