On the representation and learning of monotone triangular transport maps

Abstract

Transportation of measure provides a versatile approach for modeling complex probability distributions, with applications in density estimation, Bayesian inference, generative modeling, and beyond. Monotone triangular transport maps$\unicode{x2014}$approximations of the Knothe$\unicode{x2013}$Rosenblatt (KR) rearrangement$\unicode{x2014}$are a canonical choice for these tasks. Yet the representation and parameterization of such maps have a significant impact on their generality and expressiveness, and on properties of the optimization problem that arises in learning a map from data (e.g., via maximum likelihood estimation). We present a general framework for representing monotone triangular maps via invertible transformations of smooth functions. We establish conditions on the transformation such that the associated infinite-dimensional minimization problem has no spurious local minima, i.e., all local minima are global minima; and we show for target distributions satisfying certain tail conditions that the unique global minimizer corresponds to the KR map. Given a sample from the target, we then propose an adaptive algorithm that estimates a sparse semi-parametric approximation of the underlying KR map. We demonstrate how this framework can be applied to joint and conditional density estimation, likelihood-free inference, and structure learning of directed graphical models, with stable generalization performance across a range of sample sizes.

40 pages, 9 figures, 3 tables

References in corpus (8)

• Normalizing Flows for Probabilistic Modeling and Inference
• Coupling-based Invertible Neural Networks Are Universal Diffeomorphism Approximators
• Deep composition of tensor-trains using squared inverse Rosenblatt transports
• Prior normalization for certified likelihood-informed subspace detection of Bayesian inverse problems
• Beyond normality: Learning sparse probabilistic graphical models in the non-Gaussian setting
• Convex Potential Flows: Universal Probability Distributions with Optimal Transport and Convex Optimization
• Nonlinear dimension reduction for surrogate modeling using gradient information
• Triangular Flows for Generative Modeling: Statistical Consistency, Smoothness Classes, and Fast Rates

Cited by in corpus (6)

• Decomposing causality into its synergistic, unique, and redundant components
• Variational Bayesian Optimal Experimental Design with Normalizing Flows
• Ensemble transport smoothing. Part II: Nonlinear updates
• Ensemble transport smoothing. Part I: Unified framework
• Generative multi-scale modeling via spatial autoregressive transport maps
• Density Estimation via Measure Transport: Outlook for Applications in the Biological Sciences