LEED: Label-Free Expression Editing via Disentanglement

Abstract

Recent studies on facial expression editing have obtained very promising progress. On the other hand, existing methods face the constraint of requiring a large amount of expression labels which are often expensive and time-consuming to collect. This paper presents an innovative label-free expression editing via disentanglement (LEED) framework that is capable of editing the expression of both frontal and profile facial images without requiring any expression label. The idea is to disentangle the identity and expression of a facial image in the expression manifold, where the neutral face captures the identity attribute and the displacement between the neutral image and the expressive image captures the expression attribute. Two novel losses are designed for optimal expression disentanglement and consistent synthesis, including a mutual expression information loss that aims to extract pure expression-related features and a siamese loss that aims to enhance the expression similarity between the synthesized image and the reference image. Extensive experiments over two public facial expression datasets show that LEED achieves superior facial expression editing qualitatively and quantitatively.

arXiv commentAccepted to ECCV 2020

References in corpus (10)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild
• word2vec Explained: deriving Mikolov et al.'s negative-sampling word-embedding method
• Learning to Discover Cross-Domain Relations with Generative Adversarial Networks
• Deep Identity-aware Transfer of Facial Attributes
• ExprGAN: Facial Expression Editing with Controllable Expression Intensity
• Geometry-Contrastive GAN for Facial Expression Transfer
• Disentangled Representation Learning for 3D Face Shape