On the capacity of deep generative networks for approximating distributions

Abstract

We study the efficacy and efficiency of deep generative networks for approximating probability distributions. We prove that neural networks can transform a low-dimensional source distribution to a distribution that is arbitrarily close to a high-dimensional target distribution, when the closeness are measured by Wasserstein distances and maximum mean discrepancy. Upper bounds of the approximation error are obtained in terms of the width and depth of neural network. Furthermore, it is shown that the approximation error in Wasserstein distance grows at most linearly on the ambient dimension and that the approximation order only depends on the intrinsic dimension of the target distribution. On the contrary, when $f$-divergences are used as metrics of distributions, the approximation property is different. We show that in order to approximate the target distribution in $f$-divergences, the dimension of the source distribution cannot be smaller than the intrinsic dimension of the target distribution.

References in corpus (14)

• Generative Adversarial Text to Image Synthesis
• Generative Adversarial Network in Medical Imaging: A Review
• Generative Moment Matching Networks
• Towards Principled Methods for Training Generative Adversarial Networks
• Deep Network Approximation for Smooth Functions
• Training generative neural networks via Maximum Mean Discrepancy optimization
• Optimal approximation of continuous functions by very deep ReLU networks
• Nonlinear Approximation and (Deep) ReLU Networks
• Deep Network Approximation Characterized by Number of Neurons
• Convergence and Concentration of Empirical Measures under Wasserstein Distance in Unbounded Functional Spaces
• The phase diagram of approximation rates for deep neural networks
• On the ability of neural nets to express distributions
• A Universal Approximation Theorem of Deep Neural Networks for Expressing Probability Distributions
• How Well Generative Adversarial Networks Learn Distributions

Cited by in corpus (4)

• Wasserstein Generative Adversarial Uncertainty Quantification in Physics-Informed Neural Networks
• Approximation bounds for norm constrained neural networks with applications to regression and GANs
• An error analysis of generative adversarial networks for learning distributions
• Non-Asymptotic Error Bounds for Bidirectional GANs