ReenactGAN: Learning to Reenact Faces via Boundary Transfer

Abstract

We present a novel learning-based framework for face reenactment. The proposed method, known as ReenactGAN, is capable of transferring facial movements and expressions from monocular video input of an arbitrary person to a target person. Instead of performing a direct transfer in the pixel space, which could result in structural artifacts, we first map the source face onto a boundary latent space. A transformer is subsequently used to adapt the boundary of source face to the boundary of target face. Finally, a target-specific decoder is used to generate the reenacted target face. Thanks to the effective and reliable boundary-based transfer, our method can perform photo-realistic face reenactment. In addition, ReenactGAN is appealing in that the whole reenactment process is purely feed-forward, and thus the reenactment process can run in real-time (30 FPS on one GTX 1080 GPU). Dataset and model will be publicly available at https://wywu.github.io/projects/ReenactGAN/ReenactGAN.html

arXiv commentAccepted to ECCV 2018. Project page: https://wywu.github.io/projects/ReenactGAN/ReenactGAN.html

References in corpus (13)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• Toward Multimodal Image-to-Image Translation
• Automatic Face Reenactment
• Plug & Play Generative Networks: Conditional Iterative Generation of Images in Latent Space
• Style Aggregated Network for Facial Landmark Detection
• FaceVR: Real-Time Facial Reenactment and Eye Gaze Control in Virtual Reality
• Hallucinated-IQA: No-Reference Image Quality Assessment via Adversarial Learning
• Face Transfer with Generative Adversarial Network
• Joint Multi-view Face Alignment in the Wild
• CycleGAN Face-off
• Deep Video Portraits
• DeepCoder: Semi-parametric Variational Autoencoders for Automatic Facial Action Coding

Cited by in corpus (6)

• Generative Adversarial Networks for Image and Video Synthesis: Algorithms and Applications
• One-shot Face Reenactment
• One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing
• Multi-attentional Deepfake Detection
• CFA-Net: Controllable Face Anonymization Network with Identity Representation Manipulation
• TransGaGa: Geometry-Aware Unsupervised Image-to-Image Translation