Facial Expression Recognition Using Enhanced Deep 3D Convolutional Neural Networks

Abstract

Deep Neural Networks (DNNs) have shown to outperform traditional methods in various visual recognition tasks including Facial Expression Recognition (FER). In spite of efforts made to improve the accuracy of FER systems using DNN, existing methods still are not generalizable enough in practical applications. This paper proposes a 3D Convolutional Neural Network method for FER in videos. This new network architecture consists of 3D Inception-ResNet layers followed by an LSTM unit that together extracts the spatial relations within facial images as well as the temporal relations between different frames in the video. Facial landmark points are also used as inputs to our network which emphasize on the importance of facial components rather than the facial regions that may not contribute significantly to generating facial expressions. Our proposed method is evaluated using four publicly available databases in subject-independent and cross-database tasks and outperforms state-of-the-art methods.

To appear in 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)

References in corpus (3)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Spatio-Temporal Facial Expression Recognition Using Convolutional Neural Networks and Conditional Random Fields
• Modeling Spatial and Temporal Cues for Multi-label Facial Action Unit Detection

Cited by in corpus (23)

• Deep Facial Expression Recognition: A Survey
• Local Learning with Deep and Handcrafted Features for Facial Expression Recognition
• A Deeper Look at Facial Expression Dataset Bias
• EmpTransfo: A Multi-head Transformer Architecture for Creating Empathetic Dialog Systems
• BReG-NeXt: Facial Affect Computing Using Adaptive Residual Networks With Bounded Gradient
• Feeding Hand-Crafted Features for Enhancing the Performance of Convolutional Neural Networks
• 2D and 3D Segmentation of uncertain local collagen fiber orientations in SHG microscopy
• Facial Expression Analysis and Its Potentials in IoT Systems: A Contemporary Survey
• Cross-Domain Facial Expression Recognition: A Unified Evaluation Benchmark and Adversarial Graph Learning
• Bounded Residual Gradient Networks (BReG-Net) for Facial Affect Computing
• Responsible AI: Gender bias assessment in emotion recognition
• Video-based Facial Expression Recognition using Graph Convolutional Networks
• Inferring Dynamic Representations of Facial Actions from a Still Image
• AU-Guided Unsupervised Domain Adaptive Facial Expression Recognition
• On the Pitfalls of Learning with Limited Data: A Facial Expression Recognition Case Study
• Deep Learning-Based Strategy for Macromolecules Classification with Imbalanced Data from Cellular Electron Cryotomography
• Adversarial Graph Representation Adaptation for Cross-Domain Facial Expression Recognition
• MAFER: a Multi-resolution Approach to Facial Expression Recognition
• A Unified Multi-scale and Multi-task Learning Framework for Driver Behaviors Reasoning
• Feature Super-Resolution Based Facial Expression Recognition for Multi-scale Low-Resolution Faces
• An audiovisual and contextual approach for categorical and continuous emotion recognition in-the-wild
• AEGIS: A real-time multimodal augmented reality computer vision based system to assist facial expression recognition for individuals with autism spectrum disorder
• A Multi-resolution Approach to Expression Recognition in the Wild