Disconnected Manifold Learning for Generative Adversarial Networks

Abstract

Natural images may lie on a union of disjoint manifolds rather than one globally connected manifold, and this can cause several difficulties for the training of common Generative Adversarial Networks (GANs). In this work, we first show that single generator GANs are unable to correctly model a distribution supported on a disconnected manifold, and investigate how sample quality, mode dropping and local convergence are affected by this. Next, we show how using a collection of generators can address this problem, providing new insights into the success of such multi-generator GANs. Finally, we explain the serious issues caused by considering a fixed prior over the collection of generators and propose a novel approach for learning the prior and inferring the necessary number of generators without any supervision. Our proposed modifications can be applied on top of any other GAN model to enable learning of distributions supported on disconnected manifolds. We conduct several experiments to illustrate the aforementioned shortcoming of GANs, its consequences in practice, and the effectiveness of our proposed modifications in alleviating these issues.

NeurIPS 2018

Cited by in corpus (18)

• Robust in Practice: Adversarial Attacks on Quantum Machine Learning
• Stabilizing Invertible Neural Networks Using Mixture Models
• Black-Box Ripper: Copying black-box models using generative evolutionary algorithms
• Membership Model Inversion Attacks for Deep Networks
• Effect of The Latent Structure on Clustering with GANs
• Learning disconnected manifolds: a no GANs land
• MMGAN: Generative Adversarial Networks for Multi-Modal Distributions
• MR-GAN: Manifold Regularized Generative Adversarial Networks
• Universality Theorems for Generative Models
• On the expressivity of bi-Lipschitz normalizing flows
• Unsupervised multi-modal Styled Content Generation
• Manifold Interpolation for Large-Scale Multi-Objective Optimization via Generative Adversarial Networks
• Diverse Image Generation via Self-Conditioned GANs
• Encoder-Powered Generative Adversarial Networks
• Learning Generative Prior with Latent Space Sparsity Constraints
• AE-OT-GAN: Training GANs from data specific latent distribution
• Neural Network based Explicit Mixture Models and Expectation-maximization based Learning
• Multiclass non-Adversarial Image Synthesis, with Application to Classification from Very Small Sample