Exploiting Spatial Dimensions of Latent in GAN for Real-time Image Editing

Abstract

Generative adversarial networks (GANs) synthesize realistic images from random latent vectors. Although manipulating the latent vectors controls the synthesized outputs, editing real images with GANs suffers from i) time-consuming optimization for projecting real images to the latent vectors, ii) or inaccurate embedding through an encoder. We propose StyleMapGAN: the intermediate latent space has spatial dimensions, and a spatially variant modulation replaces AdaIN. It makes the embedding through an encoder more accurate than existing optimization-based methods while maintaining the properties of GANs. Experimental results demonstrate that our method significantly outperforms state-of-the-art models in various image manipulation tasks such as local editing and image interpolation. Last but not least, conventional editing methods on GANs are still valid on our StyleMapGAN. Source code is available at https://github.com/naver-ai/StyleMapGAN.

arXiv commentAccepted to CVPR 2021

References in corpus (15)

• Auto-Encoding Variational Bayes
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop
• Which Training Methods for GANs do actually Converge?
• Adversarial Feature Learning
• Invertible Conditional GANs for image editing
• VEEGAN: Reducing Mode Collapse in GANs using Implicit Variational Learning
• U-GAT-IT: Unsupervised Generative Attentional Networks with Adaptive Layer-Instance Normalization for Image-to-Image Translation
• Large Scale Adversarial Representation Learning
• Unsupervised Discovery of Interpretable Directions in the GAN Latent Space
• NSML: Meet the MLaaS platform with a real-world case study
• NSML: A Machine Learning Platform That Enables You to Focus on Your Models
• Megapixel Size Image Creation using Generative Adversarial Networks
• Spatially Controllable Image Synthesis with Internal Representation Collaging