Geometric variational inference

Abstract

Efficiently accessing the information contained in non-linear and high dimensional probability distributions remains a core challenge in modern statistics. Traditionally, estimators that go beyond point estimates are either categorized as Variational Inference (VI) or Markov-Chain Monte-Carlo (MCMC) techniques. While MCMC methods that utilize the geometric properties of continuous probability distributions to increase their efficiency have been proposed, VI methods rarely use the geometry. This work aims to fill this gap and proposes geometric Variational Inference (geoVI), a method based on Riemannian geometry and the Fisher information metric. It is used to construct a coordinate transformation that relates the Riemannian manifold associated with the metric to Euclidean space. The distribution, expressed in the coordinate system induced by the transformation, takes a particularly simple form that allows for an accurate variational approximation by a normal distribution. Furthermore, the algorithmic structure allows for an efficient implementation of geoVI which is demonstrated on multiple examples, ranging from low-dimensional illustrative ones to non-linear, hierarchical Bayesian inverse problems in thousands of dimensions.

arXiv comment42 pages, 18 figures, accepted by Entropy

References in corpus (10)

• Variational Inference: A Review for Statisticians
• MCMC using Hamiltonian dynamics
• Resolving nearby dust clouds
• The Galactic Faraday depth sky revisited
• Comparison of classical and Bayesian imaging in radio interferometry
• A General Metric for Riemannian Manifold Hamiltonian Monte Carlo
• Unified Radio Interferometric Calibration and Imaging with Joint Uncertainty Quantification
• Large-scale Log-determinant Computation through Stochastic Chebyshev Expansions
• Reconstructing non-repeating radio pulses with Information Field Theory
• Field dynamics inference for local and causal interactions

Cited by in corpus (25)

• A parsec-scale Galactic 3D dust map out to 1.25 kpc from the Sun
• Re-Envisioning Numerical Information Field Theory (NIFTy.re): A Library for Gaussian Processes and Variational Inference
• Bayesian radio interferometric imaging with direction-dependent calibration
• Fast and robust Bayesian Inference using Gaussian Processes with GPry
• Disentangling the Faraday rotation sky
• Spatially Coherent 3D Distributions of HI and CO in the Milky Way
• Reconstructing the Universe with Variational self-Boosted Sampling
• Bayesian self-calibration and imaging in Very Long Baseline Interferometry
• Imaging a ring-like structure and the extended jet of M87 at 86 GHz
• Information Field Theory and Artificial Intelligence
• Exploiting the diversity of modeling methods to probe systematic biases in strong lensing analyses
• Multi-Component Imaging of the Fermi Gamma-ray Sky in the Spatio-spectral Domain
• First spatio-spectral Bayesian imaging of SN1006 in X-ray
• El Gordo needs El Anzuelo: Probing the structure of cluster members with multi-band extended arcs in JWST data
• Local dark matter density from Gaia DR3 K-dwarfs using Gaussian processes
• Mid-infrared extinction toward the Galactic center
• Non-parametric Bayesian reconstruction of Galactic magnetic fields using Information Field Theory: The inclusion of line-of-sight information in ultra-high energy cosmic ray backtracking
• Interpolation techniques for reconstructing Galactic Faraday rotation
• Quantifying imperfect cognition via achieved information gain
• Bayesian Multi-wavelength Imaging of the LMC SN1987A with SRG/eROSITA
• Inferring Evidence from Nested Sampling Data via Information Field Theory
• Latent-space Field Tension for Astrophysical Component Detection An application to X-ray imaging
• Towards a Field Based Bayesian Evidence Inference from Nested Sampling Data
• Probabilistic Autoencoder using Fisher Information
• A Bayesian Method for Air-Shower Reconstruction using Information Field Theory