F3A-GAN: Facial Flow for Face Animation with Generative Adversarial Networks

Abstract

Formulated as a conditional generation problem, face animation aims at synthesizing continuous face images from a single source image driven by a set of conditional face motion. Previous works mainly model the face motion as conditions with 1D or 2D representation (e.g., action units, emotion codes, landmark), which often leads to low-quality results in some complicated scenarios such as continuous generation and largepose transformation. To tackle this problem, the conditions are supposed to meet two requirements, i.e., motion information preserving and geometric continuity. To this end, we propose a novel representation based on a 3D geometric flow, termed facial flow, to represent the natural motion of the human face at any pose. Compared with other previous conditions, the proposed facial flow well controls the continuous changes to the face. After that, in order to utilize the facial flow for face editing, we build a synthesis framework generating continuous images with conditional facial flows. To fully take advantage of the motion information of facial flows, a hierarchical conditional framework is designed to combine the extracted multi-scale appearance features from images and motion features from flows in a hierarchical manner. The framework then decodes multiple fused features back to images progressively. Experimental results demonstrate the effectiveness of our method compared to other state-of-the-art methods.

References in corpus (13)

• Conditional Generative Adversarial Nets
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• VoxCeleb2: Deep Speaker Recognition
• First Order Motion Model for Image Animation
• 3D Face Reconstruction from A Single Image Assisted by 2D Face Images in the Wild
• SiGAN: Siamese Generative Adversarial Network for Identity-Preserving Face Hallucination
• ExprGAN: Facial Expression Editing with Controllable Expression Intensity
• Deep Spatial Transformation for Pose-Guided Person Image Generation and Animation
• Unified Generative Adversarial Networks for Controllable Image-to-Image Translation
• Geometry-Contrastive GAN for Facial Expression Transfer
• Landmark Assisted CycleGAN for Cartoon Face Generation
• HUMBO: Bridging Response Generation and Facial Expression Synthesis

Cited by in corpus (1)

• Adma-GAN: Attribute-Driven Memory Augmented GANs for Text-to-Image Generation