Alleviation of Gradient Exploding in GANs: Fake Can Be Real

Abstract

In order to alleviate the notorious mode collapse phenomenon in generative adversarial networks (GANs), we propose a novel training method of GANs in which certain fake samples are considered as real ones during the training process. This strategy can reduce the gradient value that generator receives in the region where gradient exploding happens. We show the process of an unbalanced generation and a vicious circle issue resulted from gradient exploding in practical training, which explains the instability of GANs. We also theoretically prove that gradient exploding can be alleviated by penalizing the difference between discriminator outputs and fake-as-real consideration for very close real and fake samples. Accordingly, Fake-As-Real GAN (FARGAN) is proposed with a more stable training process and a more faithful generated distribution. Experiments on different datasets verify our theoretical analysis.

arXiv commentAccepted by CVPR2020

References in corpus (18)

• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• Spectral Normalization for Generative Adversarial Networks
• Self-Attention Generative Adversarial Networks
• Energy-based Generative Adversarial Network
• Which Training Methods for GANs do actually Converge?
• Large Scale Adversarial Representation Learning
• Stabilizing Training of Generative Adversarial Networks through Regularization
• Generalization and Equilibrium in Generative Adversarial Nets (GANs)
• Loss-Sensitive Generative Adversarial Networks on Lipschitz Densities
• On the regularization of Wasserstein GANs
• The Unusual Effectiveness of Averaging in GAN Training
• Improving Generalization and Stability of Generative Adversarial Networks
• On Self Modulation for Generative Adversarial Networks
• Prescribed Generative Adversarial Networks
• Generalization in Generative Adversarial Networks: A Novel Perspective from Privacy Protection
• Mixed batches and symmetric discriminators for GAN training