Kullback-Leibler Approximation for Probability Measures on Infinite Dimensional Spaces

Abstract

In a variety of applications it is important to extract information from a probability measure $\mu$ on an infinite dimensional space. Examples include the Bayesian approach to inverse problems and possibly conditioned) continuous time Markov processes. It may then be of interest to find a measure $\nu$, from within a simple class of measures, which approximates $\mu$. This problem is studied in the case where the Kullback-Leibler divergence is employed to measure the quality of the approximation. A calculus of variations viewpoint is adopted and the particular case where $\nu$ is chosen from the set of Gaussian measures is studied in detail. Basic existence and uniqueness theorems are established, together with properties of minimising sequences. Furthermore, parameterisation of the class of Gaussians through the mean and inverse covariance is introduced, the need for regularisation is explained, and a regularised minimisation is studied in detail. The calculus of variations framework resulting from this work provides the appropriate underpinning for computational algorithms.

38 pages, 2 figures

Cited by in corpus (10)

• Model-Free Data-Driven Inference in Computational Mechanics
• Scalable Information Inequalities for Uncertainty Quantification
• Variational Gaussian Approximation for Poisson Data
• Variational Bayes' method for functions with applications to some inverse problems
• Stein variational gradient descent on infinite-dimensional space and applications to statistical inverse problems
• Non-centered parametric variational Bayes' approach for hierarchical inverse problems of partial differential equations
• Spectral density of random graphs: convergence properties and application in model fitting
• Variational Characterizations of Local Entropy and Heat Regularization in Deep Learning
• Relative Entropy and Mutual Information in Gaussian Statistical Field Theory
• Optimal low-rank posterior mean and distribution approximation in linear Gaussian inverse problems on Hilbert spaces