Universal Approximation of Residual Flows in Maximum Mean Discrepancy
arXiv:2103.05793
Abstract
Normalizing flows are a class of flexible deep generative models that offer easy likelihood computation. Despite their empirical success, there is little theoretical understanding of their expressiveness. In this work, we study residual flows, a class of normalizing flows composed of Lipschitz residual blocks. We prove residual flows are universal approximators in maximum mean discrepancy. We provide upper bounds on the number of residual blocks to achieve approximation under different assumptions.
References in corpus (14)
- Variational Inference with Normalizing Flows
- On the Number of Linear Regions of Deep Neural Networks
- MADE: Masked Autoencoder for Distribution Estimation
- Residual Flows for Invertible Generative Modeling
- ResNet with one-neuron hidden layers is a Universal Approximator
- Representation Benefits of Deep Feedforward Networks
- Augmented Neural ODEs
- On the ability of neural nets to express distributions
- Sum-of-Squares Polynomial Flow
- Approximation Capabilities of Neural ODEs and Invertible Residual Networks
- Universal Lipschitz Approximation in Bounded Depth Neural Networks
- Representational aspects of depth and conditioning in normalizing flows
- Approximating Lipschitz continuous functions with GroupSort neural networks
- Constructive Universal High-Dimensional Distribution Generation through Deep ReLU Networks