Meta Auxiliary Learning for Facial Action Unit Detection

Abstract

Despite the success of deep neural networks on facial action unit (AU) detection, better performance depends on a large number of training images with accurate AU annotations. However, labeling AU is time-consuming, expensive, and error-prone. Considering AU detection and facial expression recognition (FER) are two highly correlated tasks, and facial expression (FE) is relatively easy to annotate, we consider learning AU detection and FER in a multi-task manner. However, the performance of the AU detection task cannot be always enhanced due to the negative transfer in the multi-task scenario. To alleviate this issue, we propose a Meta Auxiliary Learning method (MAL) that automatically selects highly related FE samples by learning adaptative weights for the training FE samples in a meta learning manner. The learned sample weights alleviate the negative transfer from two aspects: 1) balance the loss of each task automatically, and 2) suppress the weights of FE samples that have large uncertainties. Experimental results on several popular AU datasets demonstrate MAL consistently improves the AU detection performance compared with the state-of-the-art multi-task and auxiliary learning methods. MAL automatically estimates adaptive weights for the auxiliary FE samples according to their semantic relevance with the primary AU detection task.

References in corpus (9)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Theoretical Models of Learning to Learn
• Learning to Reweight Examples for Robust Deep Learning
• Self-supervised learning of a facial attribute embedding from video
• Self-Supervised Generalisation with Meta Auxiliary Learning
• Multi-task, multi-label and multi-domain learning with residual convolutional networks for emotion recognition
• Learning What and Where to Transfer
• SALT: Subspace Alignment as an Auxiliary Learning Task for Domain Adaptation