A Deep Learning Perspective on the Origin of Facial Expressions

Abstract

Facial expressions play a significant role in human communication and behavior. Psychologists have long studied the relationship between facial expressions and emotions. Paul Ekman et al., devised the Facial Action Coding System (FACS) to taxonomize human facial expressions and model their behavior. The ability to recognize facial expressions automatically, enables novel applications in fields like human-computer interaction, social gaming, and psychological research. There has been a tremendously active research in this field, with several recent papers utilizing convolutional neural networks (CNN) for feature extraction and inference. In this paper, we employ CNN understanding methods to study the relation between the features these computational networks are using, the FACS and Action Units (AU). We verify our findings on the Extended Cohn-Kanade (CK+), NovaEmotions and FER2013 datasets. We apply these models to various tasks and tests using transfer learning, including cross-dataset validation and cross-task performance. Finally, we exploit the nature of the FER based CNN models for the detection of micro-expressions and achieve state-of-the-art accuracy using a simple long-short-term-memory (LSTM) recurrent neural network (RNN).

References in corpus (5)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Striving for Simplicity: The All Convolutional Net
• Challenges in Representation Learning: A report on three machine learning contests
• Do Deep Neural Networks Learn Facial Action Units When Doing Expression Recognition?

Cited by in corpus (13)

• Deep Facial Expression Recognition: A Survey
• Deep Learning for Human Affect Recognition: Insights and New Developments
• Advanced local motion patterns for macro and micro facial expression recognition
• Facial Expression Recognition using Facial Landmark Detection and Feature Extraction via Neural Networks
• Predicting Player Engagement in Tom Clancy's The Division 2: A Multimodal Approach via Pixels and Gamepad Actions
• From the Lab to the Wild: Affect Modeling via Privileged Information
• Improving Facial Emotion Recognition Systems Using Gradient and Laplacian Images
• Sparse Coding of Shape Trajectories for Facial Expression and Action Recognition
• Point Adversarial Self Mining: A Simple Method for Facial Expression Recognition
• Omni-supervised Facial Expression Recognition via Distilled Data
• Towards Privacy-Preserving Affect Recognition: A Two-Level Deep Learning Architecture
• Expression Recognition Analysis in the Wild
• Emotion recognition with 4kresolution database