Fast and robust Bayesian Inference using Gaussian Processes with GPry

Abstract

We present the GPry algorithm for fast Bayesian inference of general (non-Gaussian) posteriors with a moderate number of parameters. GPry does not need any pre-training, special hardware such as GPUs, and is intended as a drop-in replacement for traditional Monte Carlo methods for Bayesian inference. Our algorithm is based on generating a Gaussian Process surrogate model of the log-posterior, aided by a Support Vector Machine classifier that excludes extreme or non-finite values. An active learning scheme allows us to reduce the number of required posterior evaluations by two orders of magnitude compared to traditional Monte Carlo inference. Our algorithm allows for parallel evaluations of the posterior at optimal locations, further reducing wall-clock times. We significantly improve performance using properties of the posterior in our active learning scheme and for the definition of the GP prior. In particular we account for the expected dynamical range of the posterior in different dimensionalities. We test our model against a number of synthetic and cosmological examples. GPry outperforms traditional Monte Carlo methods when the evaluation time of the likelihood (or the calculation of theoretical observables) is of the order of seconds; for evaluation times of over a minute it can perform inference in days that would take months using traditional methods. GPry is distributed as an open source Python package (pip install gpry) and can also be found at https://github.com/jonaselgammal/GPry.

arXiv comment39 pages, 13 figures. Submitted to JCAP

References in corpus (60)

• emcee: The MCMC Hammer
• Cosmological parameters from CMB and other data: a Monte-Carlo approach
• MultiNest: an efficient and robust Bayesian inference tool for cosmology and particle physics
• The Cosmic Linear Anisotropy Solving System (CLASS) II: Approximation schemes
• dynesty: A Dynamic Nested Sampling Package for Estimating Bayesian Posteriors and Evidences
• Multimodal nested sampling: an efficient and robust alternative to MCMC methods for astronomical data analysis
• Planck 2018 results. V. CMB power spectra and likelihoods
• The frontier of simulation-based inference
• Importance Nested Sampling and the MultiNest Algorithm
• Planck 2018 results. VIII. Gravitational lensing
• Cobaya: Code for Bayesian Analysis of hierarchical physical models
• Efficient sampling of fast and slow cosmological parameters
• PolyChord: nested sampling for cosmology
• PolyChord: next-generation nested sampling
• Dynamic nested sampling: an improved algorithm for parameter estimation and evidence calculation
• Fast likelihood-free cosmology with neural density estimators and active learning
• The BAO+BBN take on the Hubble tension
• COSMOPOWER: emulating cosmological power spectra for accelerated Bayesian inference from next-generation surveys
• The Aemulus Project II: Emulating the Halo Mass Function
• CosmoHammer: Cosmological parameter estimation with the MCMC Hammer
• The Ecological Impact of High-performance Computing in Astrophysics
• Fast cosmological parameter estimation using neural networks
• Bayesian emulator optimisation for cosmology: application to the Lyman-alpha forest
• Gravitational-wave surrogate models powered by artificial neural networks: The ANN-Sur for waveform generation
• {\sc CosmoNet}: fast cosmological parameter estimation in non-flat models using neural networks
• An Emulator for the Lyman-alpha Forest
• Deep Generative Models for Galaxy Image Simulations
• Rapid Calculation of Theoretical CMB Angular Power Spectra
• CONNECT: A neural network based framework for emulating cosmological observables and cosmological parameter inference
• Novel Method for Incorporating Model Uncertainties into Gravitational Wave Parameter Estimates
• Geometric variational inference
• Fast Cosmological Parameter Estimation from Microwave Background Temperature and Polarization Power Spectra
• Cosmological parameter estimation via iterative emulation of likelihoods
• Differentiable Strong Lensing: Uniting Gravity and Neural Nets through Differentiable Probabilistic Programming
• CosmicNet II: Emulating extended cosmologies with efficient and accurate neural networks
• Kernel-Based Emulator for the 3D Matter Power Spectrum from CLASS
• Towards constraining warm dark matter with stellar streams through neural simulation-based inference
• Multi-Fidelity Emulation for the Matter Power Spectrum using Gaussian Processes
• Real-Time Likelihood-Free Inference of Roman Binary Microlensing Events with Amortized Neural Posterior Estimation
• Robust Simulation-Based Inference in Cosmology with Bayesian Neural Networks
• CosmicNet I: Physics-driven implementation of neural networks within Boltzmann-Einstein solvers
• Euclid-era cosmology for everyone: Neural net assisted MCMC sampling for the joint 3x2 likelihood
• $\text{matryoshka}$: Halo Model Emulator for the Galaxy Power Spectrum
• Parameter Inference for Weak Lensing using Gaussian Processes and MOPED
• Lightning-Fast Gravitational Wave Parameter Inference through Neural Amortization
• LINNA: Likelihood Inference Neural Network Accelerator
• 21st Century Statistical and Computational Challenges in Astrophysics
• $\text{matryoshka}$ II: Accelerating Effective Field Theory Analyses of the Galaxy Power Spectrum
• Unbiased likelihood-free inference of the Hubble constant from light standard sirens
• Adaptive quadrature schemes for Bayesian inference via active learning
• Accurate X-ray Timing in the Presence of Systematic Biases With Simulation-Based Inference
• Accelerating cosmological inference with Gaussian processes and neural networks -- an application to LSST Y1 weak lensing and galaxy clustering
• Bayesian evidence-driven likelihood selection for sky-averaged 21-cm signal extraction
• Learning the Evolution of the Universe in N-body Simulations
• PyVBMC: Efficient Bayesian inference in Python
• Normalizing flows for random fields in cosmology
• Fast emulation of two-point angular statistics for photometric galaxy surveys
• Bayesian error propagation for neural-net based parameter inference
• Estimating Cosmological Constraints from Galaxy Cluster Abundance using Simulation-Based Inference
• Parallelized Acquisition for Active Learning using Monte Carlo Sampling

Cited by in corpus (9)

• CONNECT: A neural network based framework for emulating cosmological observables and cosmological parameter inference
• Capse.jl: efficient and auto-differentiable CMB power spectra emulation
• Halo Occupation Distribution of Emission Line Galaxies: fitting method with Gaussian Processes
• Fast and effortless computation of profile likelihoods using CONNECT
• PyVBMC: Efficient Bayesian inference in Python
• Calculating Bayesian evidence for inflationary models using CONNECT
• Accelerating LISA inference with Gaussian processes
• On the accuracy of posterior recovery with neural network emulators
• ABCMB: A Python+JAX Package for the Cosmic Microwave Background Power Spectrum