Semantic Facial Expression Editing using Autoencoded Flow

Abstract

High-level manipulation of facial expressions in images --- such as changing a smile to a neutral expression --- is challenging because facial expression changes are highly non-linear, and vary depending on the appearance of the face. We present a fully automatic approach to editing faces that combines the advantages of flow-based face manipulation with the more recent generative capabilities of Variational Autoencoders (VAEs). During training, our model learns to encode the flow from one expression to another over a low-dimensional latent space. At test time, expression editing can be done simply using latent vector arithmetic. We evaluate our methods on two applications: 1) single-image facial expression editing, and 2) facial expression interpolation between two images. We demonstrate that our method generates images of higher perceptual quality than previous VAE and flow-based methods.

References in corpus (8)

• Auto-Encoding Variational Bayes
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks
• Fully Connected Deep Structured Networks
• Semantic Image Segmentation via Deep Parsing Network
• View Synthesis by Appearance Flow
• DeepWarp: Photorealistic Image Resynthesis for Gaze Manipulation
• Towards Automatic Image Editing: Learning to See another You

Cited by in corpus (22)

• An Introduction to Variational Autoencoders
• Deep Facial Expression Recognition: A Survey
• MaskGAN: Towards Diverse and Interactive Facial Image Manipulation
• ST-GAN: Spatial Transformer Generative Adversarial Networks for Image Compositing
• Generative Adversarial Talking Head: Bringing Portraits to Life with a Weakly Supervised Neural Network
• One-shot Face Reenactment
• Photorealistic Facial Expression Synthesis by the Conditional Difference Adversarial Autoencoder
• Region Based Adversarial Synthesis of Facial Action Units
• Facelet-Bank for Fast Portrait Manipulation
• Sparsity in Variational Autoencoders
• Temporal Interpolation via Motion Field Prediction
• Autoencoder Image Interpolation by Shaping the Latent Space
• Instance-level Facial Attributes Transfer with Geometry-Aware Flow
• All-In-One: Facial Expression Transfer, Editing and Recognition Using A Single Network
• Unconstrained Facial Expression Transfer using Style-based Generator
• Automated Deepfake Detection
• Facial Expression Editing with Continuous Emotion Labels
• Learning Symmetry Consistent Deep CNNs for Face Completion
• Attribute-Driven Spontaneous Motion in Unpaired Image Translation
• Lie Group Auto-Encoder
• Joint Deep Learning of Facial Expression Synthesis and Recognition
• Learning Sparsity of Representations with Discrete Latent Variables