Variational Bayesian Monte Carlo
arXiv:1810.05558
Abstract
Many probabilistic models of interest in scientific computing and machine learning have expensive, black-box likelihoods that prevent the application of standard techniques for Bayesian inference, such as MCMC, which would require access to the gradient or a large number of likelihood evaluations. We introduce here a novel sample-efficient inference framework, Variational Bayesian Monte Carlo (VBMC). VBMC combines variational inference with Gaussian-process based, active-sampling Bayesian quadrature, using the latter to efficiently approximate the intractable integral in the variational objective. Our method produces both a nonparametric approximation of the posterior distribution and an approximate lower bound of the model evidence, useful for model selection. We demonstrate VBMC both on several synthetic likelihoods and on a neuronal model with data from real neurons. Across all tested problems and dimensions (up to ), VBMC performs consistently well in reconstructing the posterior and the model evidence with a limited budget of likelihood evaluations, unlike other methods that work only in very low dimensions. Our framework shows great promise as a novel tool for posterior and model inference with expensive, black-box likelihoods.
arXiv commentIn Advances in Neural Information Processing Systems 31 (NeurIPS 2018), pp. 8222-8232. (25 pages, 9 figures)
Cited by in corpus (8)
- A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges
- Unbiased and Efficient Log-Likelihood Estimation with Inverse Binomial Sampling
- Variational Bayesian Monte Carlo with Noisy Likelihoods
- Batch simulations and uncertainty quantification in Gaussian process surrogate approximate Bayesian computation
- Approximate Bayesian inference from noisy likelihoods with Gaussian process emulated MCMC
- Interpolating between sampling and variational inference with infinite stochastic mixtures
- Black-box density function estimation using recursive partitioning
- Inverse Gaussian Process regression for likelihood-free inference