GANs with Variational Entropy Regularizers: Applications in Mitigating the Mode-Collapse Issue

Abstract

Building on the success of deep learning, Generative Adversarial Networks (GANs) provide a modern approach to learn a probability distribution from observed samples. GANs are often formulated as a zero-sum game between two sets of functions; the generator and the discriminator. Although GANs have shown great potentials in learning complex distributions such as images, they often suffer from the mode collapse issue where the generator fails to capture all existing modes of the input distribution. As a consequence, the diversity of generated samples is lower than that of the observed ones. To tackle this issue, we take an information-theoretic approach and maximize a variational lower bound on the entropy of the generated samples to increase their diversity. We call this approach GANs with Variational Entropy Regularizers (GAN+VER). Existing remedies for the mode collapse issue in GANs can be easily coupled with our proposed variational entropy regularization. Through extensive experimentation on standard benchmark datasets, we show all the existing evaluation metrics highlighting difference of real and generated samples are significantly improved with GAN+VER.

References in corpus (22)

• Adam: A Method for Stochastic Optimization
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• Spectral Normalization for Generative Adversarial Networks
• Progressive Growing of GANs for Improved Quality, Stability, and Variation
• InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets
• Adversarial Feature Learning
• Adversarially Learned Inference
• f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization
• Unrolled Generative Adversarial Networks
• Generative Moment Matching Networks
• MaskGAN: Better Text Generation via Filling in the______
• Generative Adversarial Networks in Computer Vision: A Survey and Taxonomy
• Mode Regularized Generative Adversarial Networks
• An empirical study on evaluation metrics of generative adversarial networks
• Generalization and Equilibrium in Generative Adversarial Nets (GANs)
• AdaGAN: Boosting Generative Models
• Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distances
• Prescribed Generative Adversarial Networks
• ATFaceGAN: Single Face Image Restoration and Recognition from Atmospheric Turbulence
• Improving Neural Machine Translation with Conditional Sequence Generative Adversarial Nets
• Normalized Wasserstein Distance for Mixture Distributions with Applications in Adversarial Learning and Domain Adaptation