GANimation: Anatomically-aware Facial Animation from a Single Image

Abstract

Recent advances in Generative Adversarial Networks (GANs) have shown impressive results for task of facial expression synthesis. The most successful architecture is StarGAN, that conditions GANs generation process with images of a specific domain, namely a set of images of persons sharing the same expression. While effective, this approach can only generate a discrete number of expressions, determined by the content of the dataset. To address this limitation, in this paper, we introduce a novel GAN conditioning scheme based on Action Units (AU) annotations, which describes in a continuous manifold the anatomical facial movements defining a human expression. Our approach allows controlling the magnitude of activation of each AU and combine several of them. Additionally, we propose a fully unsupervised strategy to train the model, that only requires images annotated with their activated AUs, and exploit attention mechanisms that make our network robust to changing backgrounds and lighting conditions. Extensive evaluation show that our approach goes beyond competing conditional generators both in the capability to synthesize a much wider range of expressions ruled by anatomically feasible muscle movements, as in the capacity of dealing with images in the wild.

Accepted as oral at ECCV 2018. Code available at https://github.com/albertpumarola/GANimation. Added minor updates

References in corpus (8)

• Auto-Encoding Variational Bayes
• Conditional Generative Adversarial Nets
• Generative Adversarial Text to Image Synthesis
• Conditional Image Synthesis With Auxiliary Classifier GANs
• Invertible Conditional GANs for image editing
• Unsupervised Image-to-Image Translation Networks
• Deep Identity-aware Transfer of Facial Attributes
• Unsupervised Person Image Synthesis in Arbitrary Poses

Cited by in corpus (5)

• FakeCatcher: Detection of Synthetic Portrait Videos using Biological Signals
• Rotate-and-Render: Unsupervised Photorealistic Face Rotation from Single-View Images
• Gesture-to-Gesture Translation in the Wild via Category-Independent Conditional Maps
• Domain-Specific Mappings for Generative Adversarial Style Transfer
• PortraitGAN for Flexible Portrait Manipulation