In-Domain GAN Inversion for Real Image Editing
arXiv:2004.00049
Abstract
Recent work has shown that a variety of semantics emerge in the latent space of Generative Adversarial Networks (GANs) when being trained to synthesize images. However, it is difficult to use these learned semantics for real image editing. A common practice of feeding a real image to a trained GAN generator is to invert it back to a latent code. However, existing inversion methods typically focus on reconstructing the target image by pixel values yet fail to land the inverted code in the semantic domain of the original latent space. As a result, the reconstructed image cannot well support semantic editing through varying the inverted code. To solve this problem, we propose an in-domain GAN inversion approach, which not only faithfully reconstructs the input image but also ensures the inverted code to be semantically meaningful for editing. We first learn a novel domain-guided encoder to project a given image to the native latent space of GANs. We then propose domain-regularized optimization by involving the encoder as a regularizer to fine-tune the code produced by the encoder and better recover the target image. Extensive experiments suggest that our inversion method achieves satisfying real image reconstruction and more importantly facilitates various image editing tasks, significantly outperforming start-of-the-arts.
arXiv commentECCV 2020 camera-ready
References in corpus (12)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
- Large Scale GAN Training for High Fidelity Natural Image Synthesis
- Spectral Normalization for Generative Adversarial Networks
- Self-Attention Generative Adversarial Networks
- LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop
- BEGAN: Boundary Equilibrium Generative Adversarial Networks
- Adversarially Learned Inference
- Invertible Conditional GANs for image editing
- Semantic Hierarchy Emerges in Deep Generative Representations for Scene Synthesis
- On the "steerability" of generative adversarial networks
- Exploiting Deep Generative Prior for Versatile Image Restoration and Manipulation