Lipschitz Constrained GANs via Boundedness and Continuity

Abstract

One of the challenges in the study of Generative Adversarial Networks (GANs) is the difficulty of its performance control. Lipschitz constraint is essential in guaranteeing training stability for GANs. Although heuristic methods such as weight clipping, gradient penalty and spectral normalization have been proposed to enforce Lipschitz constraint, it is still difficult to achieve a solution that is both practically effective and theoretically provably satisfying a Lipschitz constraint. In this paper, we introduce the boundedness and continuity ($BC$) conditions to enforce the Lipschitz constraint on the discriminator functions of GANs. We prove theoretically that GANs with discriminators meeting the BC conditions satisfy the Lipschitz constraint. We present a practically very effective implementation of a GAN based on a convolutional neural network (CNN) by forcing the CNN to satisfy the $BC$ conditions (BC-GAN). We show that as compared to recent techniques including gradient penalty and spectral normalization, BC-GANs not only have better performances but also lower computational complexity.

arXiv comment20 pages, 7 figures

References in corpus (12)

• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Spectral Normalization for Generative Adversarial Networks
• Which Training Methods for GANs do actually Converge?
• Robust Large Margin Deep Neural Networks
• A Note on the Inception Score
• LR-GAN: Layered Recursive Generative Adversarial Networks for Image Generation
• Plug & Play Generative Networks: Conditional Iterative Generation of Images in Latent Space
• Lipschitz-Margin Training: Scalable Certification of Perturbation Invariance for Deep Neural Networks
• Sorting out Lipschitz function approximation
• Preventing Gradient Attenuation in Lipschitz Constrained Convolutional Networks
• Skill Rating for Generative Models

Cited by in corpus (1)

• An Improved Self-supervised GAN via Adversarial Training