The Expressive Power of a Class of Normalizing Flow Models

Abstract

Normalizing flows have received a great deal of recent attention as they allow flexible generative modeling as well as easy likelihood computation. While a wide variety of flow models have been proposed, there is little formal understanding of the representation power of these models. In this work, we study some basic normalizing flows and rigorously establish bounds on their expressive power. Our results indicate that while these flows are highly expressive in one dimension, in higher dimensions their representation power may be limited, especially when the flows have moderate depth.

arXiv comment29 pages, 7 figures; to appear on AISTATS 2020

Cited by in corpus (10)

• Ab-initio study of interacting fermions at finite temperature with neural canonical transformation
• Score-based Generative Modeling in Latent Space
• VAEBM: A Symbiosis between Variational Autoencoders and Energy-based Models
• Representational aspects of depth and conditioning in normalizing flows
• Sparse Flows: Pruning Continuous-depth Models
• Variational Monte Carlo Approach to Partial Differential Equations with Neural Networks
• Jacobian Determinant of Normalizing Flows
• Learning and Generalization in Overparameterized Normalizing Flows
• Minimax Analysis for Inverse Risk in Nonparametric Planer Invertible Regression
• Generalization of the Change of Variables Formula with Applications to Residual Flows