Two MIT students demonstrated yesterday an application for smart clocks that is able to detect the user’s emotions.To do this, the application collects data from both the device’s sensors and the speech of its users.
With this set of data, the system, created by Tuka AlHanai and Mohammad Mahdi Ghassemi, is able to classify the emotional state of a user’s speech between positive, neutral and negative.The video below shows the application running:
Among the data collected by the device are heart rate, blood pressure, temperature and level of movement of users.Other than that, the device’s microphone also captures what the user is talking about (or listening to), and uses an algorithm to analyze the emotions contained in that speech.
Emotions and safety
According to MIT, negative emotions were associated with long pauses between phrases or words and monotonous vocal tones.On the other hand, positive emotions were related to a greater variation in speech tones and more movement of the body.In addition, the system was able to relate negative emotions to short and repetitive movements and increased cardiovascular activity, in addition to certain poses such as putting a hand on the face.
As the application captures the speech of users and close people, the emotional interpretation algorithm is run locally in the test version.According to its creators, a version aimed at end consumers would have clear protocols for data usage and user consent.
Using neural networks to train the system to learn to discern between positive and negative emotions, the researchers managed to achieve an accuracy of 83% in the determination of emotions.The application can also evaluate the overall tone of the conversations by dividing them into five second passages and evaluating them individually.
According to Engadget , such an application could be very useful for people with mental conditions that make socialization difficult, such as autism or anxiety.To get to that point, however, researchers still want to develop it further.This would be done through test versions compatible with more wearable devices, such as Apple Watch.