Self-supervised Contrastive Learning of Multi-view Facial Expressions
arXiv:2108.06723 · doi:10.1145/3462244.3479955
Abstract
Facial expression recognition (FER) has emerged as an important component of human-computer interaction systems. Despite recent advancements in FER, performance often drops significantly for non-frontal facial images. We propose Contrastive Learning of Multi-view facial Expressions (CL-MEx) to exploit facial images captured simultaneously from different angles towards FER. CL-MEx is a two-step training framework. In the first step, an encoder network is pre-trained with the proposed self-supervised contrastive loss, where it learns to generate view-invariant embeddings for different views of a subject. The model is then fine-tuned with labeled data in a supervised setting. We demonstrate the performance of the proposed method on two multi-view FER datasets, KDEF and DDCF, where state-of-the-art performances are achieved. Further experiments show the robustness of our method in dealing with challenging angles and reduced amounts of labeled data.
arXiv commentAccepted by 23rd ACM International Conference on Multimodal Interaction (ICMI 2021)
References in corpus (14)
- A Simple Framework for Contrastive Learning of Visual Representations
- Representation Learning with Contrastive Predictive Coding
- Unsupervised Representation Learning by Predicting Image Rotations
- Learning Representations by Maximizing Mutual Information Across Views
- Big Self-Supervised Models are Strong Semi-Supervised Learners
- Exploring Simple Siamese Representation Learning
- Self-supervised ECG Representation Learning for Emotion Recognition
- Multi-task Self-Supervised Learning for Human Activity Detection
- Contrastive learning of global and local features for medical image segmentation with limited annotations
- Self-supervised Learning for ECG-based Emotion Recognition
- Supervised Contrastive Learning
- Instant Automated Inference of Perceived Mental Stress through Smartphone PPG and Thermal Imaging
- Detection of Maternal and Fetal Stress from the Electrocardiogram with Self-Supervised Representation Learning
- Pose-adaptive Hierarchical Attention Network for Facial Expression Recognition