Non-saturating GAN training as divergence minimization

Abstract

Non-saturating generative adversarial network (GAN) training is widely used and has continued to obtain groundbreaking results. However so far this approach has lacked strong theoretical justification, in contrast to alternatives such as f-GANs and Wasserstein GANs which are motivated in terms of approximate divergence minimization. In this paper we show that non-saturating GAN training does in fact approximately minimize a particular f-divergence. We develop general theoretical tools to compare and classify f-divergences and use these to show that the new f-divergence is qualitatively similar to reverse KL. These results help to explain the high sample quality but poor diversity often observed empirically when using this scheme.

References in corpus (12)

• Auto-Encoding Variational Bayes
• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• Spectral Normalization for Generative Adversarial Networks
• Which Training Methods for GANs do actually Converge?
• $f$-divergence Inequalities
• On Convergence and Stability of GANs
• The Numerics of GANs
• Amortised MAP Inference for Image Super-resolution
• The Mechanics of n-Player Differentiable Games
• Improved generator objectives for GANs
• Variance Regularizing Adversarial Learning
• Properties of f-divergences and f-GAN training

Cited by in corpus (2)

• Diffusion Models in Vision: A Survey
• Zero-shot Medical Image Translation via Frequency-Guided Diffusion Models