Coarse-to-Fine Gaze Redirection with Numerical and Pictorial Guidance
arXiv:2004.03064
Abstract
Gaze redirection aims at manipulating the gaze of a given face image with respect to a desired direction (i.e., a reference angle) and it can be applied to many real life scenarios, such as video-conferencing or taking group photos. However, previous work on this topic mainly suffers of two limitations: (1) Low-quality image generation and (2) Low redirection precision. In this paper, we propose to alleviate these problems by means of a novel gaze redirection framework which exploits both a numerical and a pictorial direction guidance, jointly with a coarse-to-fine learning strategy. Specifically, the coarse branch learns the spatial transformation which warps input image according to desired gaze. On the other hand, the fine-grained branch consists of a generator network with conditional residual image learning and a multi-task discriminator. This second branch reduces the gap between the previously warped image and the ground-truth image and recovers finer texture details. Moreover, we propose a numerical and pictorial guidance module~(NPG) which uses a pictorial gazemap description and numerical angles as an extra guide to further improve the precision of gaze redirection. Extensive experiments on a benchmark dataset show that the proposed method outperforms the state-of-the-art approaches in terms of both image quality and redirection precision. The code is available at https://github.com/jingjingchen777/CFGR
arXiv comment12 pages, accepted by WACV 2021
References in corpus (13)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
- Invertible Conditional GANs for image editing
- Semantic Segmentation using Adversarial Networks
- Deep Pictorial Gaze Estimation
- Generative Image Inpainting with Contextual Attention
- DRIT++: Diverse Image-to-Image Translation via Disentangled Representations
- RelGAN: Multi-Domain Image-to-Image Translation via Relative Attributes
- Multi-Channel Attention Selection GANs for Guided Image-to-Image Translation
- STGAN: A Unified Selective Transfer Network for Arbitrary Image Attribute Editing
- Cascade EF-GAN: Progressive Facial Expression Editing with Local Focuses
- GazeCorrection:Self-Guided Eye Manipulation in the wild using Self-Supervised Generative Adversarial Networks
- Unsupervised Representation Learning for Gaze Estimation