Photorealistic Facial Expression Synthesis by the Conditional Difference Adversarial Autoencoder

Abstract

Photorealistic facial expression synthesis from single face image can be widely applied to face recognition, data augmentation for emotion recognition or entertainment. This problem is challenging, in part due to a paucity of labeled facial expression data, making it difficult for algorithms to disambiguate changes due to identity and changes due to expression. In this paper, we propose the conditional difference adversarial autoencoder, CDAAE, for facial expression synthesis. The CDAAE takes a facial image of a previously unseen person and generates an image of that human face with a target emotion or facial action unit label. The CDAAE adds a feedforward path to an autoencoder structure connecting low level features at the encoder to features at the corresponding level at the decoder. It handles the problem of disambiguating changes due to identity and changes due to facial expression by learning to generate the difference between low-level features of images of the same person but with different facial expressions. The CDAAE structure can be used to generate novel expressions by combining and interpolating between facial expressions/action units within the training set. Our experimental results demonstrate that the CDAAE can preserve identity information when generating facial expression for unseen subjects more faithfully than previous approaches. This is especially advantageous when training with small databases.

Accepted by ACII2017

References in corpus (7)

• Conditional Generative Adversarial Nets
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks
• Autoencoding beyond pixels using a learned similarity metric
• Age Progression/Regression by Conditional Adversarial Autoencoder
• Semantic Facial Expression Editing using Autoencoded Flow
• Towards Automatic Image Editing: Learning to See another You

Cited by in corpus (5)

• Deep Facial Expression Recognition: A Survey
• Dynamic Facial Expression Generation on Hilbert Hypersphere with Conditional Wasserstein Generative Adversarial Nets
• Geometry-Contrastive GAN for Facial Expression Transfer
• Adversarial Training in Affective Computing and Sentiment Analysis: Recent Advances and Perspectives
• Talking Head Generation with Audio and Speech Related Facial Action Units