Facial Response Characteristics Are Correlated to More Complex Emotions

Generally, there are six emotions that are considered the basics for cognitive analysis - anger, happiness, fear, surprise, disgust and sadness. Computer recognition programs have focused on correlating only these six emotions with facial expressions, but a group of researchers from Ohio State University have fine-tuned this to detect 21 different emotions. Their work was sponsored by the National Institutes of Health (NIH) and summarized in the recent Proceedings of the National Academy of Sciences.

Using 230 volunteers, the team took photographs of the volunteer's response to various verbal cues, such as detecting a foul odor or getting unexpected positive news. With 5,000 total images in hand, the team chose a series of specific facial areas for analysis, such as the outer edges of the lips and eyebrows, tagged them, and analyzed them to look for common reactions across different volunteers for the same verbal cue.

Using a tool for analyzing body language known as the Facial Action Coding System and searching for correlations, the team was able to expand on the standard six emotions by identifying "compound emotions"-blends of the six standard emotions and the accompanying facial expressions.

For example, let's consider "happily surprised." Happiness correlates well to drawing up of the cheeks and smiling, and surprise correlates well to the classic wide-eyed, open-mouthed response. "Happily surprised" volunteers consistently retained all characteristics of both emotions-drawn up cheeks, wide-open eyes, and a mouth smiling but partially open.

Seemingly incompatible emotions were also distinguished, such as "happily disgusted." This is a response to something gross or unsettling, but yet so humorous that you can't help but laugh, combining features of happiness with the scrunched-up nose and eyes that indicate disgust.

The team drew the analogy to a painter who previously could only paint with primary colors. Just as a painter can produce richer and more realistic paintings with a larger color palette, cognitive scientists can use these findings to produce a more in-depth understanding of how the mind recognizes and analyzes more complex emotions.

On a more clinical level, the goal is help people with disorders through greater understanding of the process that our brains use in detecting emotions through facial expressions. People with autism, who generally cannot grasp the emotions of others through their facial expressions, have a different or faulty algorithm for this task. Through understanding the differences in the autistic brain's processing versus an average brain's processing, it may be possible in the future to target treatments.

Similarly, people suffering from post-traumatic stress disorder (PTSD) that are more highly sensitive to emotions such as anger and fear are going to exhibit different processing of those two emotions in their brains compared to the average brain. If the chemical or neurological pathways are better understood, treatments might be possible.

This study doesn't enable work of this nature-it has been going on for some time now-but it does allow future work to have a greater resolution and clarity by incorporating more complex emotions into the study. We look forward to seeing the results that cognitive researchers will "paint" with their new palette.