Facial expression and attributes recognition based on multi-task learning of lightweight neural networks

Abstract

In this paper, the multi-task learning of lightweight convolutional neural networks is studied for face identification and classification of facial attributes (age, gender, ethnicity) trained on cropped faces without margins. The necessity to fine-tune these networks to predict facial expressions is highlighted. Several models are presented based on MobileNet, EfficientNet and RexNet architectures. It was experimentally demonstrated that they lead to near state-of-the-art results in age, gender and race recognition on the UTKFace dataset and emotion classification on the AffectNet dataset. Moreover, it is shown that the usage of the trained models as feature extractors of facial regions in video frames leads to 4.5% higher accuracy than the previously known state-of-the-art single models for the AFEW and the VGAF datasets from the EmotiW challenges. The models and source code are publicly available at https://github.com/HSE-asavchenko/face-emotion-recognition.

14 pages, 3 figures, accepted at IEEE SISY 2021

References in corpus (10)

• AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild
• Distract Your Attention: Multi-head Cross Attention Network for Facial Expression Recognition
• Leveraging Recent Advances in Deep Learning for Audio-Visual Emotion Recognition
• Compacting, Picking and Growing for Unforgetting Continual Learning
• Efficient Facial Representations for Age, Gender and Identity Recognition in Organizing Photo Albums using Multi-output CNN
• Learning to Amend Facial Expression Representation via De-albino and Affinity
• Exploiting Emotional Dependencies with Graph Convolutional Networks for Facial Expression Recognition
• Noisy Student Training using Body Language Dataset Improves Facial Expression Recognition
• CAKE: Compact and Accurate K-dimensional representation of Emotion
• Computational Emotion Analysis From Images: Recent Advances and Future Directions

Cited by in corpus (6)

• Distract Your Attention: Multi-head Cross Attention Network for Facial Expression Recognition
• Headset: Human emotion awareness under partial occlusions multimodal dataset
• Nonverbal Immediacy Analysis in Education: A Multimodal Computational Model
• Detecting Interlocutor Confusion in Situated Human-Avatar Dialogue: A Pilot Study
• Multimodal End-to-End Group Emotion Recognition using Cross-Modal Attention
• Affect-DML: Context-Aware One-Shot Recognition of Human Affect using Deep Metric Learning