An error analysis of generative adversarial networks for learning distributions

Abstract

This paper studies how well generative adversarial networks (GANs) learn probability distributions from finite samples. Our main results establish the convergence rates of GANs under a collection of integral probability metrics defined through Hölder classes, including the Wasserstein distance as a special case. We also show that GANs are able to adaptively learn data distributions with low-dimensional structures or have Hölder densities, when the network architectures are chosen properly. In particular, for distributions concentrated around a low-dimensional set, we show that the learning rates of GANs do not depend on the high ambient dimension, but on the lower intrinsic dimension. Our analysis is based on a new oracle inequality decomposing the estimation error into the generator and discriminator approximation error and the statistical error, which may be of independent interest.

References in corpus (23)

• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Generative Adversarial Text to Image Synthesis
• Generative Adversarial Network in Medical Imaging: A Review
• Progressive Growing of GANs for Improved Quality, Stability, and Variation
• Improved Training of Wasserstein GANs
• Generative Moment Matching Networks
• On Convergence and Stability of GANs
• Generalization and Equilibrium in Generative Adversarial Nets (GANs)
• Deep Network Approximation for Smooth Functions
• Optimal approximation of continuous functions by very deep ReLU networks
• On the regularization of Wasserstein GANs
• Nonlinear Approximation and (Deep) ReLU Networks
• Improving Generalization and Stability of Generative Adversarial Networks
• Nonlinear Approximation via Compositions
• Adaptive Approximation and Generalization of Deep Neural Network with Intrinsic Dimensionality
• The phase diagram of approximation rates for deep neural networks
• Improving the Improved Training of Wasserstein GANs: A Consistency Term and Its Dual Effect
• On the ability of neural nets to express distributions
• Approximability of Discriminators Implies Diversity in GANs
• A Universal Approximation Theorem of Deep Neural Networks for Expressing Probability Distributions
• Nonparametric Density Estimation under Adversarial Losses
• How Well Generative Adversarial Networks Learn Distributions
• On the capacity of deep generative networks for approximating distributions

Cited by in corpus (4)

• Approximation bounds for norm constrained neural networks with applications to regression and GANs
• Convergence of flow-based generative models via proximal gradient descent in Wasserstein space
• Statistically Optimal Generative Modeling with Maximum Deviation from the Empirical Distribution
• Just Least Squares: Binary Compressive Sampling with Low Generative Intrinsic Dimension