MGRR-Net: Multi-level Graph Relational Reasoning Network for Facial Action Units Detection

Abstract

The Facial Action Coding System (FACS) encodes the action units (AUs) in facial images, which has attracted extensive research attention due to its wide use in facial expression analysis. Many methods that perform well on automatic facial action unit (AU) detection primarily focus on modeling various types of AU relations between corresponding local muscle areas, or simply mining global attention-aware facial features, however, neglect the dynamic interactions among local-global features. We argue that encoding AU features just from one perspective may not capture the rich contextual information between regional and global face features, as well as the detailed variability across AUs, because of the diversity in expression and individual characteristics. In this paper, we propose a novel Multi-level Graph Relational Reasoning Network (termed MGRR-Net) for facial AU detection. Each layer of MGRR-Net performs a multi-level (i.e., region-level, pixel-wise and channel-wise level) feature learning. While the region-level feature learning from local face patches features via graph neural network can encode the correlation across different AUs, the pixel-wise and channel-wise feature learning via graph attention network can enhance the discrimination ability of AU features from global face features. The fused features from the three levels lead to improved AU discriminative ability. Extensive experiments on DISFA and BP4D AU datasets show that the proposed approach achieves superior performance than the state-of-the-art methods.

20 pages, 5 figures, 8 tables;

References in corpus (9)

• Semi-Supervised Classification with Graph Convolutional Networks
• Short and Long Range Relation Based Spatio-Temporal Transformer for Micro-Expression Recognition
• J$\hat{\text{A}}$A-Net: Joint Facial Action Unit Detection and Face Alignment via Adaptive Attention
• AU R-CNN: Encoding Expert Prior Knowledge into R-CNN for Action Unit Detection
• GeoConv: Geodesic Guided Convolution for Facial Action Unit Recognition
• Deep Multi-Center Learning for Face Alignment
• Deep Reasoning with Knowledge Graph for Social Relationship Understanding
• Contrastive Learning of Person-independent Representations for Facial Action Unit Detection
• Multi-Order Networks for Action Unit Detection

Cited by in corpus (1)

• Towards End-to-End Explainable Facial Action Unit Recognition via Vision-Language Joint Learning