Deep Adaptive Attention for Joint Facial Action Unit Detection and Face Alignment

Abstract

Facial action unit (AU) detection and face alignment are two highly correlated tasks since facial landmarks can provide precise AU locations to facilitate the extraction of meaningful local features for AU detection. Most existing AU detection works often treat face alignment as a preprocessing and handle the two tasks independently. In this paper, we propose a novel end-to-end deep learning framework for joint AU detection and face alignment, which has not been explored before. In particular, multi-scale shared features are learned firstly, and high-level features of face alignment are fed into AU detection. Moreover, to extract precise local features, we propose an adaptive attention learning module to refine the attention map of each AU adaptively. Finally, the assembled local features are integrated with face alignment features and global features for AU detection. Experiments on BP4D and DISFA benchmarks demonstrate that our framework significantly outperforms the state-of-the-art methods for AU detection.

This paper has been accepted by ECCV 2018

References in corpus (1)

• Conditional Adversarial Synthesis of 3D Facial Action Units

Cited by in corpus (16)

• J$\hat{\text{A}}$A-Net: Joint Facial Action Unit Detection and Face Alignment via Adaptive Attention
• Learning Multi-dimensional Edge Feature-based AU Relation Graph for Facial Action Unit Recognition
• Facial Action Unit Detection Using Attention and Relation Learning
• Domain-Incremental Continual Learning for Mitigating Bias in Facial Expression and Action Unit Recognition
• Facial Action Unit Detection via Adaptive Attention and Relation
• Facial Expression Restoration Based on Improved Graph Convolutional Networks
• Region Based Adversarial Synthesis of Facial Action Units
• Contrastive Learning of Person-independent Representations for Facial Action Unit Detection
• Explicit Facial Expression Transfer via Fine-Grained Representations
• Your "Attention" Deserves Attention: A Self-Diversified Multi-Channel Attention for Facial Action Analysis
• Towards End-to-End Explainable Facial Action Unit Recognition via Vision-Language Joint Learning
• Multi-Modal Learning for AU Detection Based on Multi-Head Fused Transformers
• Unconstrained Facial Action Unit Detection via Latent Feature Domain
• A PCA based Keypoint Tracking Approach to Automated Facial Expressions Encoding
• Cross-subject Action Unit Detection with Meta Learning and Transformer-based Relation Modeling
• FreeAvatar: Robust 3D Facial Animation Transfer by Learning an Expression Foundation Model