Inference via low-dimensional couplings

Abstract

We investigate the low-dimensional structure of deterministic transformations between random variables, i.e., transport maps between probability measures. In the context of statistics and machine learning, these transformations can be used to couple a tractable "reference" measure (e.g., a standard Gaussian) with a target measure of interest. Direct simulation from the desired measure can then be achieved by pushing forward reference samples through the map. Yet characterizing such a map---e.g., representing and evaluating it---grows challenging in high dimensions. The central contribution of this paper is to establish a link between the Markov properties of the target measure and the existence of low-dimensional couplings, induced by transport maps that are sparse and/or decomposable. Our analysis not only facilitates the construction of transformations in high-dimensional settings, but also suggests new inference methodologies for continuous non-Gaussian graphical models. For instance, in the context of nonlinear state-space models, we describe new variational algorithms for filtering, smoothing, and sequential parameter inference. These algorithms can be understood as the natural generalization---to the non-Gaussian case---of the square-root Rauch-Tung-Striebel Gaussian smoother.

arXiv comment78 pages, 25 figures

References in corpus (15)

• Density estimation using Real NVP
• Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm
• Bayesian Inference with Optimal Maps
• Data Assimilation: A Mathematical Introduction
• Learning in Implicit Generative Models
• Likelihood-informed dimension reduction for nonlinear inverse problems
• An introduction to sampling via measure transport
• Iterative Gaussianization: from ICA to Random Rotations
• Gibbs flow for approximate transport with applications to Bayesian computation
• Stein Variational Adaptive Importance Sampling
• Higher order Quasi-Monte Carlo integration for Bayesian Estimation
• Beyond normality: Learning sparse probabilistic graphical models in the non-Gaussian setting
• Warp Bridge Sampling: The Next Generation
• A general spline representation for nonparametric and semiparametric density estimates using diffeomorphisms
• Bayesian Lasso Posterior Sampling via Parallelized Measure Transport

Cited by in corpus (5)

• Approximating the Quadratic Transportation Metric in Near-Linear Time
• Beyond normality: Learning sparse probabilistic graphical models in the non-Gaussian setting
• VAE-KRnet and its applications to variational Bayes
• Augmented KRnet for density estimation and approximation
• Particle filters for high-dimensional geoscience applications: a review