Improving Inversion and Generation Diversity in StyleGAN using a Gaussianized Latent Space

Abstract

Modern Generative Adversarial Networks are capable of creating artificial, photorealistic images from latent vectors living in a low-dimensional learned latent space. It has been shown that a wide range of images can be projected into this space, including images outside of the domain that the generator was trained on. However, while in this case the generator reproduces the pixels and textures of the images, the reconstructed latent vectors are unstable and small perturbations result in significant image distortions. In this work, we propose to explicitly model the data distribution in latent space. We show that, under a simple nonlinear operation, the data distribution can be modeled as Gaussian and therefore expressed using sufficient statistics. This yields a simple Gaussian prior, which we use to regularize the projection of images into the latent space. The resulting projections lie in smoother and better behaved regions of the latent space, as shown using interpolation performance for both real and generated images. Furthermore, the Gaussian model of the distribution in latent space allows us to investigate the origins of artifacts in the generator output, and provides a method for reducing these artifacts while maintaining diversity of the generated images.

References in corpus (6)

• Auto-Encoding Variational Bayes
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• Adversarial Feature Learning
• Interpreting the Latent Space of GANs for Semantic Face Editing

Cited by in corpus (9)

• GANSpace: Discovering Interpretable GAN Controls
• Using latent space regression to analyze and leverage compositionality in GANs
• InfinityGAN: Towards Infinite-Pixel Image Synthesis
• FreeStyleGAN: Free-view Editable Portrait Rendering with the Camera Manifold
• Prior Image-Constrained Reconstruction using Style-Based Generative Models
• In&Out : Diverse Image Outpainting via GAN Inversion
• Ensembling with Deep Generative Views
• Reconstructing Interpretable Features in Computational Super-Resolution microscopy via Regularized Latent Search
• StyleGAN of All Trades: Image Manipulation with Only Pretrained StyleGAN