DeepWarp: Photorealistic Image Resynthesis for Gaze Manipulation

Abstract

In this work, we consider the task of generating highly-realistic images of a given face with a redirected gaze. We treat this problem as a specific instance of conditional image generation and suggest a new deep architecture that can handle this task very well as revealed by numerical comparison with prior art and a user study. Our deep architecture performs coarse-to-fine warping with an additional intensity correction of individual pixels. All these operations are performed in a feed-forward manner, and the parameters associated with different operations are learned jointly in the end-to-end fashion. After learning, the resulting neural network can synthesize images with manipulated gaze, while the redirection angle can be selected arbitrarily from a certain range and provided as an input to the network.

arXiv commentFixed typos, 14 + 2 + 2 pages, ECCV 2016

References in corpus (7)

• Adam: A Method for Stochastic Optimization
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks
• DRAW: A Recurrent Neural Network For Image Generation
• Deep Convolutional Inverse Graphics Network
• Texture Synthesis Using Convolutional Neural Networks
• Towards Automatic Image Editing: Learning to See another You

Cited by in corpus (7)

• HeadOn: Real-time Reenactment of Human Portrait Videos
• Semantic Facial Expression Editing using Autoencoded Flow
• ST-GAN: Spatial Transformer Generative Adversarial Networks for Image Compositing
• Real-Time Video Super-Resolution with Spatio-Temporal Networks and Motion Compensation
• WarpGAN: Automatic Caricature Generation
• Dual In-painting Model for Unsupervised Gaze Correction and Animation in the Wild
• Learning Symmetry Consistent Deep CNNs for Face Completion