MaGNET: Uniform Sampling from Deep Generative Network Manifolds Without Retraining

Abstract

Deep Generative Networks (DGNs) are extensively employed in Generative Adversarial Networks (GANs), Variational Autoencoders (VAEs), and their variants to approximate the data manifold and distribution. However, training samples are often distributed in a non-uniform fashion on the manifold, due to costs or convenience of collection. For example, the CelebA dataset contains a large fraction of smiling faces. These inconsistencies will be reproduced when sampling from the trained DGN, which is not always preferred, e.g., for fairness or data augmentation. In response, we develop MaGNET, a novel and theoretically motivated latent space sampler for any pre-trained DGN, that produces samples uniformly distributed on the learned manifold. We perform a range of experiments on various datasets and DGNs, e.g., for the state-of-the-art StyleGAN2 trained on FFHQ dataset, uniform sampling via MaGNET increases distribution precision and recall by 4.1\% \& 3.0\% and decreases gender bias by 41.2\%, without requiring labels or retraining. As uniform distribution does not imply uniform semantic distribution, we also explore separately how semantic attributes of generated samples vary under MaGNET sampling.

arXiv commentICLR Accepted version, 28 pages, 23 figures

References in corpus (22)

• Auto-Encoding Variational Bayes
• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• Spectral Normalization for Generative Adversarial Networks
• Neural Discrete Representation Learning
• NICE: Non-linear Independent Components Estimation
• Energy-based Generative Adversarial Network
• On the Number of Linear Regions of Deep Neural Networks
• Density estimation using Real NVP
• Unrolled Generative Adversarial Networks
• NVAE: A Deep Hierarchical Variational Autoencoder
• On Convergence and Stability of GANs
• Assessing Generative Models via Precision and Recall
• Variational Recurrent Auto-Encoders
• Sylvester Normalizing Flows for Variational Inference
• Hyperspherical Variational Auto-Encoders
• Improving Variational Auto-Encoders using Householder Flow
• Theory and Experiments on Vector Quantized Autoencoders
• Gaussian Mixture Generative Adversarial Networks for Diverse Datasets, and the Unsupervised Clustering of Images
• Improving the Fairness of Deep Generative Models without Retraining
• FairFaceGAN: Fairness-aware Facial Image-to-Image Translation
• Discriminator optimal transport
• Max-Affine Spline Insights into Deep Generative Networks