Universal Joint Approximation of Manifolds and Densities by Simple Injective Flows

Abstract

We study approximation of probability measures supported on $n$-dimensional manifolds embedded in ${\mathbb{R}}^m$ by injective flows -- neural networks composed of invertible flows and injective layers. We show that in general, injective flows between ${\mathbb{R}}^n$ and ${\mathbb{R}}^m$ universally approximate measures supported on images of extendable embeddings, which are a subset of standard embeddings: when the embedding dimension m is small, topological obstructions may preclude certain manifolds as admissible targets. When the embedding dimension is sufficiently large, $m\ge 3n+1$, we use an argument from algebraic topology known as the clean trick to prove that the topological obstructions vanish and injective flows universally approximate any differentiable embedding. Along the way we show that the studied injective flows admit efficient projections on the range, and that their optimality can be established "in reverse," resolving a conjecture made in Brehmer and Cranmer 2020.

26 pages, 5 figures

References in corpus (13)

• NICE: Non-linear Independent Components Estimation
• Compressed Sensing using Generative Models
• The Reversible Residual Network: Backpropagation Without Storing Activations
• Coupling-based Invertible Neural Networks Are Universal Diffeomorphism Approximators
• Sum-of-Squares Polynomial Flow
• Flows for simultaneous manifold learning and density estimation
• Benchmarking Invertible Architectures on Inverse Problems
• Cubic-Spline Flows
• Deep Probabilistic Imaging: Uncertainty Quantification and Multi-modal Solution Characterization for Computational Imaging
• Globally Injective ReLU Networks
• When and How Can Deep Generative Models be Inverted?
• Normalizing Flows Across Dimensions
• Trumpets: Injective Flows for Inference and Inverse Problems

Cited by in corpus (1)

• Continuous Generative Neural Networks: A Wavelet-Based Architecture in Function Spaces