Skip to Main Content
 

Global Search Box

 
 
 
 

Files

File List


Dissertation_Du (1).pdf (5.32 MB)

ETD Abstract Container

Abstract Header

A Computational Model of the Production and Perception of Facial Expressions of Basic and Compound Emotions

Du, Shichuan
http://rave.ohiolink.edu/etdc/view?acc_num=osu1405989041

Abstract Details

2014, Doctor of Philosophy, Ohio State University, Electrical and Computer Engineering.
In this dissertation, we investigate the production and recognition of facial expressions of emotion and propose a computational model of human perception. We first show that in addition to the six basic facial expressions (happiness, sadness, fear, anger, surprise and disgust), 15 compound facial expressions can be produced by integrating two basic emotions, e.g., happily surprised and fearfully surprised. The production involves consistent muscle movements in 230 human participants within each emotion category while different emotions employ different groups of muscles. We hypothesize that this production results in visually distinctive expressions. This is tested by the proposed computational model, using second-order information, i.e., intra-facial-component distances, also referred to as configural features. This information is extracted from face shapes, which are the collection of the coordinates of fiducial points delineating the contour of each facial component (e.g., eyes, brows, nose, mouth and entire face). This model is shown to distinguish among the 21 facial expressions (6 basic and 15 compound) highly accurately (72.09%, chance=4.55%). This suggests that the production of facial expressions of emotion may lie far beyond the six basic emotions. These compound emotions, along with the six basic emotions, provide a new platform to test various hypotheses of human cognition and computer vision systems. To investigate human recognition of facial expressions of emotion, human participants were tested with facial expressions of the basic and compound emotions at 5 different resolutions (from 240 by 160 to 15 by 10 pixels) and asked to select one emotion label that matches the image from 7 or 8 choices. Results show that recognition is mostly impaired when the image resolution goes below 30 by 20 pixels; happiness, surprise, happily surprised and happily disgusted were robustly recognized with high accuracies across resolutions, whereas the other emotions were recognized with less accuracies at all resolutions. The asymmetry of the confusion tables is consistent over the different image resolutions and cannot be explained by the similarity of muscle activation. These results suggest that the cognitive space is defined to achieve a constant recognition for a variety of image resolutions (or viewing distances). To further challenge the human visual system and study its processing of facial expressions, we then investigated the minimum exposure times required to successfully classify the six basic facial expressions of emotion as seen at different image resolutions. The results suggest a consistent hierarchical analysis of these facial expressions regardless of the resolution of the stimuli. Happiness and surprise can be recognized after very short exposure times (10 to 20 ms), even at low resolutions. Sadness and disgust are recognized in between (70 to 310 ms). Fear and anger are recognized the slowest (100 to 450 ms), even in high-resolution images, suggesting a later computation. The minimum exposure time required for successful classification of each facial expression correlates with the ability of a human subject to identify it correctly at low resolutions. These results suggest a fast, early computation of expressions represented mostly by low spatial frequencies or global configural cues and a later, slower process for those categories requiring a more fine-grained analysis of the image. Lastly, the shape-based computational model was tested in the task of classifying the basic and compound emotions at those 5 resolutions described above. The performance of the computation model is highly similar to the human performance (r = 0.89) on average across emotions, and both of their performance are robust from 240 by 160 to 30 by 20 pixels. This suggests that the configural features from shapes carry discriminant information of these emotions in a wide range of resolutions. The mutual information between the shape and its emotion category is also consistent with human performance. Hence, these results indicate that configural/shape features play an important role in perception of facial expressions of emotion.
Aleix Martinez (Advisor)
111 p.
Cognitive Psychology; Electrical Engineering; Psychology
facial expressions of emotion, computational modeling, human visual system, compound emotions, cognitive space of emotion

Recommended Citations

Citations

  • Du, S. (2014). A Computational Model of the Production and Perception of Facial Expressions of Basic and Compound Emotions [Doctoral dissertation, Ohio State University]. OhioLINK Electronic Theses and Dissertations Center. http://rave.ohiolink.edu/etdc/view?acc_num=osu1405989041

    APA Style (7th edition)

  • Du, Shichuan. A Computational Model of the Production and Perception of Facial Expressions of Basic and Compound Emotions. 2014. Ohio State University, Doctoral dissertation. OhioLINK Electronic Theses and Dissertations Center, http://rave.ohiolink.edu/etdc/view?acc_num=osu1405989041.

    MLA Style (8th edition)

  • Du, Shichuan. "A Computational Model of the Production and Perception of Facial Expressions of Basic and Compound Emotions." Doctoral dissertation, Ohio State University, 2014. http://rave.ohiolink.edu/etdc/view?acc_num=osu1405989041

    Chicago Manual of Style (17th edition)

osu1405989041
836
© 2014, all rights reserved.
This open access ETD is published by The Ohio State University and OhioLINK.