Conditional Adversarial Synthesis of 3D Facial Action Units

Abstract

Employing deep learning-based approaches for fine-grained facial expression analysis, such as those involving the estimation of Action Unit (AU) intensities, is difficult due to the lack of a large-scale dataset of real faces with sufficiently diverse AU labels for training. In this paper, we consider how AU-level facial image synthesis can be used to substantially augment such a dataset. We propose an AU synthesis framework that combines the well-known 3D Morphable Model (3DMM), which intrinsically disentangles expression parameters from other face attributes, with models that adversarially generate 3DMM expression parameters conditioned on given target AU labels, in contrast to the more conventional approach of generating facial images directly. In this way, we are able to synthesize new combinations of expression parameters and facial images from desired AU labels. Extensive quantitative and qualitative results on the benchmark DISFA dataset demonstrate the effectiveness of our method on 3DMM facial expression parameter synthesis and data augmentation for deep learning-based AU intensity estimation.

References in corpus (8)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Conditional Generative Adversarial Nets
• InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets
• Adversarial Autoencoders
• ExprGAN: Facial Expression Editing with Controllable Expression Intensity
• Recent Progress of Face Image Synthesis
• CNN-based Real-time Dense Face Reconstruction with Inverse-rendered Photo-realistic Face Images

Cited by in corpus (5)

• Deep Adaptive Attention for Joint Facial Action Unit Detection and Face Alignment
• Relation Modeling with Graph Convolutional Networks for Facial Action Unit Detection
• GeoConv: Geodesic Guided Convolution for Facial Action Unit Recognition
• Generative Adversarial Talking Head: Bringing Portraits to Life with a Weakly Supervised Neural Network
• Region Based Adversarial Synthesis of Facial Action Units