Going Deeper in Facial Expression Recognition using Deep Neural Networks

Abstract

Automated Facial Expression Recognition (FER) has remained a challenging and interesting problem. Despite efforts made in developing various methods for FER, existing approaches traditionally lack generalizability when applied to unseen images or those that are captured in wild setting. Most of the existing approaches are based on engineered features (e.g. HOG, LBPH, and Gabor) where the classifier's hyperparameters are tuned to give best recognition accuracies across a single database, or a small collection of similar databases. Nevertheless, the results are not significant when they are applied to novel data. This paper proposes a deep neural network architecture to address the FER problem across multiple well-known standard face datasets. Specifically, our network consists of two convolutional layers each followed by max pooling and then four Inception layers. The network is a single component architecture that takes registered facial images as the input and classifies them into either of the six basic or the neutral expressions. We conducted comprehensive experiments on seven publically available facial expression databases, viz. MultiPIE, MMI, CK+, DISFA, FERA, SFEW, and FER2013. The results of proposed architecture are comparable to or better than the state-of-the-art methods and better than traditional convolutional neural networks and in both accuracy and training time.

To be appear in IEEE Winter Conference on Applications of Computer Vision (WACV), 2016 {Accepted in first round submission}

References in corpus (6)

• Caffe: Convolutional Architecture for Fast Feature Embedding
• Going Deeper with Convolutions
• DeepID3: Face Recognition with Very Deep Neural Networks
• ExpressionBot: An Emotive Lifelike Robotic Face for Face-to-Face Communication
• Bidirectional Warping of Active Appearance Model
• eBear: An Expressive Bear-Like Robot

Cited by in corpus (68)

• AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild
• Deep Facial Expression Recognition: A Survey
• Spatial-Temporal Recurrent Neural Network for Emotion Recognition
• Facial Expression Recognition Using Enhanced Deep 3D Convolutional Neural Networks
• Facial Emotion Recognition: State of the Art Performance on FER2013
• A Deeper Look at Facial Expression Dataset Bias
• Review and Comparison of Commonly Used Activation Functions for Deep Neural Networks
• Facial Expression Recognition Using a Hybrid CNN-SIFT Aggregator
• Facial Expression Recognition from World Wild Web
• Deep generative-contrastive networks for facial expression recognition
• SADRNet: Self-Aligned Dual Face Regression Networks for Robust 3D Dense Face Alignment and Reconstruction
• Spatio-Temporal Facial Expression Recognition Using Convolutional Neural Networks and Conditional Random Fields
• Learning to Amend Facial Expression Representation via De-albino and Affinity
• Advanced local motion patterns for macro and micro facial expression recognition
• FaceNet2ExpNet: Regularizing a Deep Face Recognition Net for Expression Recognition
• Semantic Neighborhood-Aware Deep Facial Expression Recognition
• BReG-NeXt: Facial Affect Computing Using Adaptive Residual Networks With Bounded Gradient
• Multi-Branch Deep Radial Basis Function Networks for Facial Emotion Recognition
• Facial Expression Recognition with Deep Learning
• Dynamic Multi-Task Learning for Face Recognition with Facial Expression
• Peak-Piloted Deep Network for Facial Expression Recognition
• Deep Covariance Descriptors for Facial Expression Recognition
• Facial Affect Estimation in the Wild Using Deep Residual and Convolutional Networks
• Probabilistic Attribute Tree in Convolutional Neural Networks for Facial Expression Recognition
• Facial Expression Recognition at the Edge: CPU vs GPU vs VPU vs TPU
• Facial Emotion Recognition using Convolutional Neural Networks
• Privacy-Preserving Deep Inference for Rich User Data on The Cloud
• From Facial Expression Recognition to Interpersonal Relation Prediction
• Your "Attention" Deserves Attention: A Self-Diversified Multi-Channel Attention for Facial Action Analysis
• Multi-Region Ensemble Convolutional Neural Network for Facial Expression Recognition
• Cross-Domain Facial Expression Recognition: A Unified Evaluation Benchmark and Adversarial Graph Learning
• Bounded Residual Gradient Networks (BReG-Net) for Facial Affect Computing
• A Fine-Grained Facial Expression Database for End-to-End Multi-Pose Facial Expression Recognition
• Learning to Augment Expressions for Few-shot Fine-grained Facial Expression Recognition
• Human Expression Recognition using Facial Shape Based Fourier Descriptors Fusion
• Differential Generative Adversarial Networks: Synthesizing Non-linear Facial Variations with Limited Number of Training Data
• Mirror Ritual: Human-Machine Co-Construction of Emotion
• A New Terrain in HCI: Emotion Recognition Interface using Biometric Data for an Immersive VR Experience
• Deep Multi-Facial Patches Aggregation Network For Facial Expression Recognition
• Uncertainty-Aware Label Refinement on Hypergraphs for Personalized Federated Facial Expression Recognition
• Video-based Facial Expression Recognition using Graph Convolutional Networks
• Disentanglement for Discriminative Visual Recognition
• On the Pitfalls of Learning with Limited Data: A Facial Expression Recognition Case Study
• AU-Guided Unsupervised Domain Adaptive Facial Expression Recognition
• Adversarial Graph Representation Adaptation for Cross-Domain Facial Expression Recognition
• Complex Facial Expression Recognition Using Deep Knowledge Distillation of Basic Features
• EXPERTNet Exigent Features Preservative Network for Facial Expression Recognition
• An optimized Capsule-LSTM model for facial expression recognition with video sequences
• Lossless Attention in Convolutional Networks for Facial Expression Recognition in the Wild
• Deep neural networks for collaborative learning analytics: Evaluating team collaborations using student gaze point prediction
• Hyperparameters optimization for Deep Learning based emotion prediction for Human Robot Interaction
• FaceLiveNet+: A Holistic Networks For Face Authentication Based On Dynamic Multi-task Convolutional Neural Networks
• AU-Expression Knowledge Constrained Representation Learning for Facial Expression Recognition
• Facial Expressions Recognition with Convolutional Neural Networks
• How far generated data can impact Neural Networks performance?
• Point Adversarial Self Mining: A Simple Method for Facial Expression Recognition
• Omni-supervised Facial Expression Recognition via Distilled Data
• Convolutional Neural Network for emotion recognition to assist psychiatrists and psychologists during the COVID-19 pandemic: experts opinion
• Deep Multi-Facial patches Aggregation Network for Expression Classification from Face Images
• Unsupervised Deep Representations for Learning Audience Facial Behaviors
• Feature Extraction via Recurrent Random Deep Ensembles and its Application in Gruop-level Happiness Estimation
• It All Matters: Reporting Accuracy, Inference Time and Power Consumption for Face Emotion Recognition on Embedded Systems
• Landmark-Aware and Part-based Ensemble Transfer Learning Network for Facial Expression Recognition from Static images
• Deep learning for identification and face, gender, expression recognition under constraints
• Learning Stylized Character Expressions from Humans
• Event-Level Emotion Recognition in the Wild Using Deep Facial Expression Analysis
• Domain Adaptation for Facial Expression Classifier via Domain Discrimination and Gradient Reversal
• Joint Deep Learning of Facial Expression Synthesis and Recognition