Improving GAN Training via Binarized Representation Entropy (BRE) Regularization

Abstract

We propose a novel regularizer to improve the training of Generative Adversarial Networks (GANs). The motivation is that when the discriminator D spreads out its model capacity in the right way, the learning signals given to the generator G are more informative and diverse. These in turn help G to explore better and discover the real data manifold while avoiding large unstable jumps due to the erroneous extrapolation made by D. Our regularizer guides the rectifier discriminator D to better allocate its model capacity, by encouraging the binary activation patterns on selected internal layers of D to have a high joint entropy. Experimental results on both synthetic data and real datasets demonstrate improvements in stability and convergence speed of the GAN training, as well as higher sample quality. The approach also leads to higher classification accuracies in semi-supervised learning.

arXiv commentPublished as a conference paper at the 6th International Conference on Learning Representations (ICLR 2018)

References in corpus (12)

• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Densely Connected Convolutional Networks
• Improved Training of Wasserstein GANs
• Improved Techniques for Training GANs
• Energy-based Generative Adversarial Network
• BEGAN: Boundary Equilibrium Generative Adversarial Networks
• Adversarial Feature Learning
• Adversarially Learned Inference
• Deep multi-scale video prediction beyond mean square error
• A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Models
• On the Expressive Power of Deep Neural Networks
• On the Joint Entropy of $d$-Wise-Independent Variables

Cited by in corpus (1)

• Prescribed Generative Adversarial Networks