Relation Modeling with Graph Convolutional Networks for Facial Action Unit Detection

Abstract

Most existing AU detection works considering AU relationships are relying on probabilistic graphical models with manually extracted features. This paper proposes an end-to-end deep learning framework for facial AU detection with graph convolutional network (GCN) for AU relation modeling, which has not been explored before. In particular, AU related regions are extracted firstly, latent representations full of AU information are learned through an auto-encoder. Moreover, each latent representation vector is feed into GCN as a node, the connection mode of GCN is determined based on the relationships of AUs. Finally, the assembled features updated through GCN are concatenated for AU detection. Extensive experiments on BP4D and DISFA benchmarks demonstrate that our framework significantly outperforms the state-of-the-art methods for facial AU detection. The proposed framework is also validated through a series of ablation studies.

Accepted by MMM2020

References in corpus (10)

• Semi-Supervised Classification with Graph Convolutional Networks
• Spectral Networks and Locally Connected Networks on Graphs
• Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering
• Deep Convolutional Networks on Graph-Structured Data
• Convolutional Networks on Graphs for Learning Molecular Fingerprints
• Learning Convolutional Neural Networks for Graphs
• Diffusion-Convolutional Neural Networks
• Gated Graph Sequence Neural Networks
• Conditional Adversarial Synthesis of 3D Facial Action Units
• Bayesian Semi-supervised Learning with Graph Gaussian Processes

Cited by in corpus (14)

• Graph-Based Deep Learning for Medical Diagnosis and Analysis: Past, Present and Future
• Learning Multi-dimensional Edge Feature-based AU Relation Graph for Facial Action Unit Recognition
• Facial Action Unit Detection via Adaptive Attention and Relation
• Facial Action Unit Detection by Adaptively Constraining Self-Attention and Causally Deconfounding Sample
• Your "Attention" Deserves Attention: A Self-Diversified Multi-Channel Attention for Facial Action Analysis
• Objective Class-based Micro-Expression Recognition through Simultaneous Action Unit Detection and Feature Aggregation
• Towards End-to-End Explainable Facial Action Unit Recognition via Vision-Language Joint Learning
• MER-GCN: Micro Expression Recognition Based on Relation Modeling with Graph Convolutional Network
• Two-stage Temporal Modelling Framework for Video-based Depression Recognition using Graph Representation
• Cross-subject Action Unit Detection with Meta Learning and Transformer-based Relation Modeling
• Self-Supervised Regional and Temporal Auxiliary Tasks for Facial Action Unit Recognition
• Computational efficient deep neural network with difference attention maps for facial action unit detection
• Multi-Level Adaptive Region of Interest and Graph Learning for Facial Action Unit Recognition
• Deep graph learning for semi-supervised classification