Approximation and Convergence Properties of Generative Adversarial Learning

Abstract

Generative adversarial networks (GAN) approximate a target data distribution by jointly optimizing an objective function through a "two-player game" between a generator and a discriminator. Despite their empirical success, however, two very basic questions on how well they can approximate the target distribution remain unanswered. First, it is not known how restricting the discriminator family affects the approximation quality. Second, while a number of different objective functions have been proposed, we do not understand when convergence to the global minima of the objective function leads to convergence to the target distribution under various notions of distributional convergence. In this paper, we address these questions in a broad and unified setting by defining a notion of adversarial divergences that includes a number of recently proposed objective functions. We show that if the objective function is an adversarial divergence with some additional conditions, then using a restricted discriminator family has a moment-matching effect. Additionally, we show that for objective functions that are strict adversarial divergences, convergence in the objective function implies weak convergence, thus generalizing previous results.

References in corpus (3)

• Generative Moment Matching Networks
• Training generative neural networks via Maximum Mean Discrepancy optimization
• Generative Models and Model Criticism via Optimized Maximum Mean Discrepancy

Cited by in corpus (13)

• Ten Years of Generative Adversarial Nets (GANs): A survey of the state-of-the-art
• A survey on GANs for computer vision: Recent research, analysis and taxonomy
• 2-Wasserstein Approximation via Restricted Convex Potentials with Application to Improved Training for GANs
• Some Theoretical Insights into Wasserstein GANs
• A likelihood approach to nonparametric estimation of a singular distribution using deep generative models
• Designing GANs: A Likelihood Ratio Approach
• Generalized Energy Based Models
• Deconstructing Generative Adversarial Networks
• Convergence and Sample Complexity of SGD in GANs
• Training Wasserstein GANs without gradient penalties
• Non-Asymptotic Error Bounds for Bidirectional GANs
• Bridging the Gap Between $f$-GANs and Wasserstein GANs
• High-Dimensional Non-Parametric Density Estimation in Mixed Smooth Sobolev Spaces