Dynamic Facial Expression Generation on Hilbert Hypersphere with Conditional Wasserstein Generative Adversarial Nets

Abstract

In this work, we propose a novel approach for generating videos of the six basic facial expressions given a neutral face image. We propose to exploit the face geometry by modeling the facial landmarks motion as curves encoded as points on a hypersphere. By proposing a conditional version of manifold-valued Wasserstein generative adversarial network (GAN) for motion generation on the hypersphere, we learn the distribution of facial expression dynamics of different classes, from which we synthesize new facial expression motions. The resulting motions can be transformed to sequences of landmarks and then to images sequences by editing the texture information using another conditional Generative Adversarial Network. To the best of our knowledge, this is the first work that explores manifold-valued representations with GAN to address the problem of dynamic facial expression generation. We evaluate our proposed approach both quantitatively and qualitatively on two public datasets; Oulu-CASIA and MUG Facial Expression. Our experimental results demonstrate the effectiveness of our approach in generating realistic videos with continuous motion, realistic appearance and identity preservation. We also show the efficiency of our framework for dynamic facial expressions generation, dynamic facial expression transfer and data augmentation for training improved emotion recognition models.

References in corpus (14)

• Adam: A Method for Stochastic Optimization
• Conditional Generative Adversarial Nets
• TensorFlow: A system for large-scale machine learning
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Fully Convolutional Networks for Semantic Segmentation
• Generating Videos with Scene Dynamics
• Action-Conditional Video Prediction using Deep Networks in Atari Games
• Decomposing Motion and Content for Natural Video Sequence Prediction
• Learning to Generate Long-term Future via Hierarchical Prediction
• Age Progression/Regression by Conditional Adversarial Autoencoder
• ExprGAN: Facial Expression Editing with Controllable Expression Intensity
• Deep Covariance Descriptors for Facial Expression Recognition
• Every Smile is Unique: Landmark-Guided Diverse Smile Generation
• Photorealistic Facial Expression Synthesis by the Conditional Difference Adversarial Autoencoder

Cited by in corpus (5)

• 4D Facial Expression Diffusion Model
• Facial Expression Video Generation Based-On Spatio-temporal Convolutional GAN: FEV-GAN
• All-In-One: Facial Expression Transfer, Editing and Recognition Using A Single Network
• An Efficient Integration of Disentangled Attended Expression and Identity FeaturesFor Facial Expression Transfer andSynthesis
• Improving Generative Adversarial Networks with Local Coordinate Coding