Sampling for Inference in Probabilistic Models with Fast Bayesian Quadrature

Abstract

We propose a novel sampling framework for inference in probabilistic models: an active learning approach that converges more quickly (in wall-clock time) than Markov chain Monte Carlo (MCMC) benchmarks. The central challenge in probabilistic inference is numerical integration, to average over ensembles of models or unknown (hyper-)parameters (for example to compute the marginal likelihood or a partition function). MCMC has provided approaches to numerical integration that deliver state-of-the-art inference, but can suffer from sample inefficiency and poor convergence diagnostics. Bayesian quadrature techniques offer a model-based solution to such problems, but their uptake has been hindered by prohibitive computation costs. We introduce a warped model for probabilistic integrands (likelihoods) that are known to be non-negative, permitting a cheap active learning scheme to optimally select sample locations. Our algorithm is demonstrated to offer faster convergence (in seconds) relative to simple Monte Carlo and annealed importance sampling on both synthetic and real-world examples.

References in corpus (1)

• Bayesian Optimal Active Search and Surveying

Cited by in corpus (19)

• Variational Inference for Gaussian Process Modulated Poisson Processes
• Frank-Wolfe Bayesian Quadrature: Probabilistic Integration with Theoretical Guarantees
• Variational Bayesian Monte Carlo with Noisy Likelihoods
• Exploiting gradients and Hessians in Bayesian optimization and Bayesian quadrature
• Adaptive quadrature schemes for Bayesian inference via active learning
• Knowing The What But Not The Where in Bayesian Optimization
• Batch simulations and uncertainty quantification in Gaussian process surrogate approximate Bayesian computation
• Control functionals for Monte Carlo integration
• PyVBMC: Efficient Bayesian inference in Python
• Active Multi-Information Source Bayesian Quadrature
• BINOCULARS for Efficient, Nonmyopic Sequential Experimental Design
• Approximate Bayesian inference from noisy likelihoods with Gaussian process emulated MCMC
• Overcoming Intensity Limits for Long-Distance Quantum Key Distribution
• Bayesian Probabilistic Numerical Integration with Tree-Based Models
• Marginalising over Stationary Kernels with Bayesian Quadrature
• Gaussian Process Sampling and Optimization with Approximate Upper and Lower Bounds
• Design of Experiments for Verifying Biomolecular Networks
• A Locally Adaptive Bayesian Cubature Method
• Black Box Probabilistic Numerics