Effect of The Latent Structure on Clustering with GANs

Abstract

Generative adversarial networks (GANs) have shown remarkable success in generation of data from natural data manifolds such as images. In several scenarios, it is desirable that generated data is well-clustered, especially when there is severe class imbalance. In this paper, we focus on the problem of clustering in generated space of GANs and uncover its relationship with the characteristics of the latent space. We derive from first principles, the necessary and sufficient conditions needed to achieve faithful clustering in the GAN framework: (i) presence of a multimodal latent space with adjustable priors, (ii) existence of a latent space inversion mechanism and (iii) imposition of the desired cluster priors on the latent space. We also identify the GAN models in the literature that partially satisfy these conditions and demonstrate the importance of all the components required, through ablative studies on multiple real world image datasets. Additionally, we describe a procedure to construct a multimodal latent space which facilitates learning of cluster priors with sparse supervision.

References in corpus (16)

• Auto-Encoding Variational Bayes
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• Spectral Normalization for Generative Adversarial Networks
• Progressive Growing of GANs for Improved Quality, Stability, and Variation
• Semi-Supervised Learning with Deep Generative Models
• Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations
• Adversarial Feature Learning
• Adversarially Learned Inference
• VEEGAN: Reducing Mode Collapse in GANs using Implicit Variational Learning
• InfoVAE: Information Maximizing Variational Autoencoders
• Unsupervised and Semi-supervised Learning with Categorical Generative Adversarial Networks
• Discovering Hidden Factors of Variation in Deep Networks
• Do GANs actually learn the distribution? An empirical study
• GP-GAN: Towards Realistic High-Resolution Image Blending
• Disconnected Manifold Learning for Generative Adversarial Networks

Cited by in corpus (2)

• Improving ClusterGAN Using Self-Augmented Information Maximization of Disentangling Latent Spaces
• Stein Latent Optimization for Generative Adversarial Networks