Multi-task, multi-label and multi-domain learning with residual convolutional networks for emotion recognition

Abstract

Automated emotion recognition in the wild from facial images remains a challenging problem. Although recent advances in Deep Learning have supposed a significant breakthrough in this topic, strong changes in pose, orientation and point of view severely harm current approaches. In addition, the acquisition of labeled datasets is costly, and current state-of-the-art deep learning algorithms cannot model all the aforementioned difficulties. In this paper, we propose to apply a multi-task learning loss function to share a common feature representation with other related tasks. Particularly we show that emotion recognition benefits from jointly learning a model with a detector of facial Action Units (collective muscle movements). The proposed loss function addresses the problem of learning multiple tasks with heterogeneously labeled data, improving previous multi-task approaches. We validate the proposal using two datasets acquired in non controlled environments, and an application to predict compound facial emotion expressions.

Preprint submitted to IJCV

References in corpus (4)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• A Survey on Multi-Task Learning
• Spatiotemporal Residual Networks for Video Action Recognition
• EmotioNet Challenge: Recognition of facial expressions of emotion in the wild

Cited by in corpus (8)

• Deep Facial Expression Recognition: A Survey
• Deep Multi-task Learning for Facial Expression Recognition and Synthesis Based on Selective Feature Sharing
• A Unified Multi-scale and Multi-task Learning Framework for Driver Behaviors Reasoning
• Facial Emotion Recognition with Noisy Multi-task Annotations
• Facial Emotion Recognition: A multi-task approach using deep learning
• Landmark-Aware and Part-based Ensemble Transfer Learning Network for Facial Expression Recognition from Static images
• Beyond without Forgetting: Multi-Task Learning for Classification with Disjoint Datasets
• Meta Auxiliary Learning for Facial Action Unit Detection