Adaptive quadrature schemes for Bayesian inference via active learning

Abstract

Numerical integration and emulation are fundamental topics across scientific fields. We propose novel adaptive quadrature schemes based on an active learning procedure. We consider an interpolative approach for building a surrogate posterior density, combining it with Monte Carlo sampling methods and other quadrature rules. The nodes of the quadrature are sequentially chosen by maximizing a suitable acquisition function, which takes into account the current approximation of the posterior and the positions of the nodes. This maximization does not require additional evaluations of the true posterior. We introduce two specific schemes based on Gaussian and Nearest Neighbors (NN) bases. For the Gaussian case, we also provide a novel procedure for fitting the bandwidth parameter, in order to build a suitable emulator of a density function. With both techniques, we always obtain a positive estimation of the marginal likelihood (a.k.a., Bayesian evidence). An equivalent importance sampling interpretation is also described, which allows the design of extended schemes. Several theoretical results are provided and discussed. Numerical results show the advantage of the proposed approach, including a challenging inference problem in an astronomic dynamical model, with the goal of revealing the number of planets orbiting a star.

arXiv commentKeywords: Numerical integration; emulation; Monte Carlo methods; Bayesian quadrature; experimental design; active learning

References in corpus (19)

• Layered Adaptive Importance Sampling
• Improved Adaptive Rejection Metropolis Sampling Algorithms
• Bayesian Re-analysis of the Gliese 581 Exoplanet System
• Accelerating Asymptotically Exact MCMC for Computationally Intensive Models via Local Approximations
• Marginal likelihood computation for model selection and hypothesis testing: an extensive review
• Active emulation of computer codes with Gaussian processes -- Application to remote sensing
• Importance Gaussian Quadrature
• Sampling for Inference in Probabilistic Models with Fast Bayesian Quadrature
• Integral approximation by kernel smoothing
• Query Efficient Posterior Estimation in Scientific Experiments via Bayesian Active Learning
• Discovery of WASP-113b and WASP-114b, two inflated hot-Jupiters with contrasting densities
• Sequential Kernel Herding: Frank-Wolfe Optimization for Particle Filtering
• Adaptive Independent Sticky MCMC algorithms
• Two Jovian planets around the giant star HD202696. A growing population of packed massive planetary pairs around massive stars?
• Convergence Guarantees for Adaptive Bayesian Quadrature Methods
• Adaptive sampling-based quadrature rules for efficient Bayesian prediction
• Moving Target Monte Carlo
• Tree pyramidal adaptive importance sampling
• Optimal Design Emulators: A Point Process Approach

Cited by in corpus (6)

• Fast and robust Bayesian Inference using Gaussian Processes with GPry
• Sequential Bayesian experimental design for calibration of expensive simulation models
• A survey of Monte Carlo methods for noisy and costly densities with application to reinforcement learning and ABC
• Target-aware Bayesian inference via generalized thermodynamic integration
• Deep Importance Sampling based on Regression for Model Inversion and Emulation
• Bayesian Probabilistic Numerical Integration with Tree-Based Models