Face Translation between Images and Videos using Identity-aware CycleGAN
arXiv:1712.00971
Abstract
This paper presents a new problem of unpaired face translation between images and videos, which can be applied to facial video prediction and enhancement. In this problem there exist two major technical challenges: 1) designing a robust translation model between static images and dynamic videos, and 2) preserving facial identity during image-video translation. To address such two problems, we generalize the state-of-the-art image-to-image translation network (Cycle-Consistent Adversarial Networks) to the image-to-video/video-to-image translation context by exploiting a image-video translation model and an identity preservation model. In particular, we apply the state-of-the-art Wasserstein GAN technique to the setting of image-video translation for better convergence, and we meanwhile introduce a face verificator to ensure the identity. Experiments on standard image/video face datasets demonstrate the effectiveness of the proposed model in both terms of qualitative and quantitative evaluations.
References in corpus (13)
- Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
- Progressive Growing of GANs for Improved Quality, Stability, and Variation
- Improved Training of Wasserstein GANs
- Energy-based Generative Adversarial Network
- Generating Videos with Scene Dynamics
- BEGAN: Boundary Equilibrium Generative Adversarial Networks
- Learning Spatiotemporal Features with 3D Convolutional Networks
- A Light CNN for Deep Face Representation with Noisy Labels
- Deep Learning Face Attributes in the Wild
- Stacked Generative Adversarial Networks
- Multi-View Image Generation from a Single-View
- Attribute-Guided Face Generation Using Conditional CycleGAN
- Generate Identity-Preserving Faces by Generative Adversarial Networks