Bayesian Inference with Optimal Maps

Abstract

We present a new approach to Bayesian inference that entirely avoids Markov chain simulation, by constructing a map that pushes forward the prior measure to the posterior measure. Existence and uniqueness of a suitable measure-preserving map is established by formulating the problem in the context of optimal transport theory. We discuss various means of explicitly parameterizing the map and computing it efficiently through solution of an optimization problem, exploiting gradient information from the forward model when possible. The resulting algorithm overcomes many of the computational bottlenecks associated with Markov chain Monte Carlo. Advantages of a map-based representation of the posterior include analytical expressions for posterior moments and the ability to generate arbitrary numbers of independent posterior samples without additional likelihood evaluations or forward solves. The optimization approach also provides clear convergence criteria for posterior approximation and facilitates model selection through automatic evaluation of the marginal likelihood. We demonstrate the accuracy and efficiency of the approach on nonlinear inverse problems of varying dimension, involving the inference of parameters appearing in ordinary and partial differential equations.

66 pages, 26 figures. Minor revisions and improvements throughout. Published in Journal of Computational Physics

References in corpus (1)

• MCMC using Hamiltonian dynamics

Cited by in corpus (78)

• An introduction to sampling via measure transport
• Parameter Identification in a Probabilistic Setting
• Principal component analysis and sparse polynomial chaos expansions for global sensitivity analysis and model calibration: application to urban drainage simulation
• Transport map accelerated Markov chain Monte Carlo
• Large-Scale Optimal Transport and Mapping Estimation
• Deterministic Mean-field Ensemble Kalman Filtering
• Data Assimilation: The Schrödinger Perspective
• Spectral likelihood expansions for Bayesian inference
• Sequential Bayesian optimal experimental design via approximate dynamic programming
• Optimal transport mapping via input convex neural networks
• On the representation and learning of monotone triangular transport maps
• Gibbs flow for approximate transport with applications to Bayesian computation
• Geophysical Inversion and Optimal Transport
• Rates of Estimation of Optimal Transport Maps using Plug-in Estimators via Barycentric Projections
• Diffeomorphic density matching by optimal information transport
• Deep composition of tensor-trains using squared inverse Rosenblatt transports
• Transport-based analysis, modeling, and learning from signal and data distributions
• Expectation Propagation for Nonlinear Inverse Problems -- with an Application to Electrical Impedance Tomography
• 2-Wasserstein Approximation via Restricted Convex Potentials with Application to Improved Training for GANs
• Parameter estimation by implicit sampling
• A multiscale strategy for Bayesian inference using transport maps
• Bayesian model inversion using stochastic spectral embedding
• Some models are useful, but how do we know which ones? Towards a unified Bayesian model taxonomy
• Continuous Regularized Wasserstein Barycenters
• Particle-based Energetic Variational Inference
• Greedy inference with structure-exploiting lazy maps
• Low-Rank Separated Representation Surrogates of High-Dimensional Stochastic Functions: Application in Bayesian Inference
• Variational Bayesian Optimal Experimental Design with Normalizing Flows
• A McKean optimal transportation perspective on Feynman-Kac formulae with application to data assimilation
• Inference via low-dimensional couplings
• Beyond normality: Learning sparse probabilistic graphical models in the non-Gaussian setting
• Quantum Wasserstein distance based on an optimization over separable states
• Probabilistic learning inference of boundary value problem with uncertainties based on Kullback-Leibler divergence under implicit constraints
• Quantum Wasserstein isometries on the qubit state space
• Latent Simplex Position Model: High Dimensional Multi-view Clustering with Uncertainty Quantification
• Kernel embedding of maps for sequential Bayesian inference: The variational mapping particle filter
• Calibration of multi-physics computational models using Bayesian networks
• Ensemble filter techniques for intermittent data assimilation - a survey
• Variational Inference for Nonlinear Inverse Problems via Neural Net Kernels: Comparison to Bayesian Neural Networks, Application to Topology Optimization
• A general spline representation for nonparametric and semiparametric density estimates using diffeomorphisms
• On the Wasserstein median of probability measures
• Learning to solve Bayesian inverse problems: An amortized variational inference approach using Gaussian and Flow guides
• Bayesian nonparametric generative modeling of large multivariate non-Gaussian spatial fields
• Diffeomorphic random sampling using optimal information transport
• Conditional deep surrogate models for stochastic, high-dimensional, and multi-fidelity systems
• Annealed Flow Transport Monte Carlo
• Ensemble transport smoothing. Part II: Nonlinear updates
• Partial inversion of the elliptic operator to speed up computation of likelihood in Bayesian inference
• Tractable Fully Bayesian Inference via Convex Optimization and Optimal Transport Theory
• Iterative Construction of Gaussian Process Surrogate Models for Bayesian Inference
• Ensemble transport smoothing. Part I: Unified framework
• Optimal Transport Filtering with Particle Reweighing in Finance
• NEO: Non Equilibrium Sampling on the Orbit of a Deterministic Transform
• Quantification of Model Uncertainty in RANS Simulations: A Review
• NF-iSAM: Incremental Smoothing and Mapping via Normalizing Flows
• Distortion estimates for approximate Bayesian inference
• Particle filters for high-dimensional geoscience applications: a review
• MALA-within-Gibbs samplers for high-dimensional distributions with sparse conditional structure
• Iterative importance sampling algorithms for parameter estimation
• Goal-oriented optimal approximations of Bayesian linear inverse problems
• Quasi-Newton Quasi-Monte Carlo for variational Bayes
• Sparse Variational Bayesian Approximations for Nonlinear Inverse Problems: applications in nonlinear elastography
• Sampling, feasibility, and priors in Bayesian estimation
• Stein variational reduced basis Bayesian inversion
• Algorithms for Kullback-Leibler Approximation of Probability Measures in Infinite Dimensions
• Construction and Analysis of Posterior Matching in Arbitrary Dimensions via Optimal Transport
• Stability of Gibbs Posteriors from the Wasserstein Loss for Bayesian Full Waveform Inversion
• Bayesian inverse problems with $l_1$ priors: a Randomize-then-Optimize approach
• An improved implicit sampling for Bayesian inverse problems of multi-term time fractional multiscale diffusion models
• Quantile Propagation for Wasserstein-Approximate Gaussian Processes
• Optimal Transportation Methods in Nonlinear Filtering: The feedback particle filter
• Ensemble Transport Filter via Optimized Maximum Mean Discrepancy
• Multilevel Stein variational gradient descent with applications to Bayesian inverse problems
• Density Estimation via Measure Transport: Outlook for Applications in the Biological Sciences
• Sequential Ensemble Transform for Bayesian Inverse Problems
• A Linear Transportation ${\mathrm{L}}^p$ Distance for Pattern Recognition
• Functional Gaussian processes for regression with linear PDE models
• Ensemble-based implicit sampling for Bayesian inverse problems with non-Gaussian priors