Variational Bayes with Synthetic Likelihood

Abstract

Synthetic likelihood is an attractive approach to likelihood-free inference when an approximately Gaussian summary statistic for the data, informative for inference about the parameters, is available. The synthetic likelihood method derives an approximate likelihood function from a plug-in normal density estimate for the summary statistic, with plug-in mean and covariance matrix obtained by Monte Carlo simulation from the model. In this article, we develop alternatives to Markov chain Monte Carlo implementations of Bayesian synthetic likelihoods with reduced computational overheads. Our approach uses stochastic gradient variational inference methods for posterior approximation in the synthetic likelihood context, employing unbiased estimates of the log likelihood. We compare the new method with a related likelihood free variational inference technique in the literature, while at the same time improving the implementation of that approach in a number of ways. These new algorithms are feasible to implement in situations which are challenging for conventional approximate Bayesian computation (ABC) methods, in terms of the dimensionality of the parameter and summary statistic.

30 pages; 7 figures

References in corpus (17)

• Auto-Encoding Variational Bayes
• ADADELTA: An Adaptive Learning Rate Method
• Stochastic Backpropagation and Approximate Inference in Deep Generative Models
• The pseudo-marginal approach for efficient Monte Carlo computations
• A Comparative Review of Dimension Reduction Methods in Approximate Bayesian Computation
• Variational Bayesian Inference with Stochastic Search
• GPS-ABC: Gaussian Process Surrogate Approximate Bayesian Computation
• Bayesian Optimization for Likelihood-Free Inference of Simulator-Based Statistical Models
• Fixed-Form Variational Posterior Approximation through Stochastic Linear Regression
• Accelerating ABC methods using Gaussian processes
• Gaussian variational approximation with sparse precision matrices
• Technical Note: Approximate Bayesian parameterization of a process-based tropical forest model
• Pre-processing for approximate Bayesian computation in image analysis
• Bayesian model comparison with un-normalised likelihoods
• Automatic Variational ABC
• Extending approximate Bayesian computation methods to high dimensions via a Gaussian copula model
• Variational Bayes with Intractable Likelihood

Cited by in corpus (14)

• Flexible statistical inference for mechanistic models of neural dynamics
• Sequential Neural Likelihood: Fast Likelihood-free Inference with Autoregressive Flows
• Likelihood-free MCMC with Amortized Approximate Ratio Estimators
• Approximating Bayes in the 21st Century
• A guide to state-space modeling of ecological time series
• Multifidelity multilevel Monte Carlo to accelerate approximate Bayesian parameter inference for partially observed stochastic processes
• Approximating the Likelihood in Approximate Bayesian Computation
• Neural Density Estimation and Likelihood-free Inference
• A synthetic likelihood-based Laplace approximation for efficient design of biological processes
• Robust Bayesian Synthetic Likelihood via a Semi-Parametric Approach
• Analytic natural gradient updates for Cholesky factor in Gaussian variational approximation
• Confidence Sets and Hypothesis Testing in a Likelihood-Free Inference Setting
• Likelihood-free approximate Gibbs sampling
• An approach for finding fully Bayesian optimal designs using normal-based approximations to loss functions