Sequentially guided MCMC proposals for synthetic likelihoods and correlated synthetic likelihoods

Abstract

Synthetic likelihood (SL) is a strategy for parameter inference when the likelihood function is analytically or computationally intractable. In SL, the likelihood function of the data is replaced by a multivariate Gaussian density over summary statistics of the data. SL requires simulation of many replicate datasets at every parameter value considered by a sampling algorithm, such as Markov chain Monte Carlo (MCMC), making the method computationally-intensive. We propose two strategies to alleviate the computational burden. First, we introduce an algorithm producing a proposal distribution that is sequentially tuned and made conditional to data, thus it rapidly \textit{guides} the proposed parameters towards high posterior density regions. In our experiments, a small number of iterations of our algorithm is enough to rapidly locate high density regions, which we use to initialize one or several chains that make use of off-the-shelf adaptive MCMC methods. Our "guided" approach can also be potentially used with MCMC samplers for approximate Bayesian computation (ABC). Second, we exploit strategies borrowed from the correlated pseudo-marginal MCMC literature, to improve the chains mixing in a SL framework. Moreover, our methods enable inference for challenging case studies, when the posterior is multimodal and when the chain is initialised in low posterior probability regions of the parameter space, where standard samplers failed. To illustrate the advantages stemming from our framework we consider five benchmark examples, including estimation of parameters for a cosmological model and a stochastic model with highly non-Gaussian summary statistics.

arXiv comment37 pages; 16 figures. revised multimodal case study added

References in corpus (31)

• Exploring the Expansion History of the Universe
• The No-U-Turn Sampler: Adaptively Setting Path Lengths in Hamiltonian Monte Carlo
• The pseudo-marginal approach for efficient Monte Carlo computations
• Adaptive approximate Bayesian computation
• The Supernova Legacy Survey 3-year sample: Type Ia Supernovae photometric distances and cosmological constraints
• Probing the dark energy: methods and strategies
• A Comparative Review of Dimension Reduction Methods in Approximate Bayesian Computation
• SNANA: A Public Software Package for Supernova Analysis
• Phenomenological parameterization of quintessence
• $\Lambda$CDM Cosmology for Astronomers
• Bayesian Optimization for Likelihood-Free Inference of Simulator-Based Statistical Models
• Dark energy equation of state parameter and its variation at low redshifts
• The dark energy equation of state
• Sequential Neural Likelihood: Fast Likelihood-free Inference with Autoregressive Flows
• astroABC: An Approximate Bayesian Computation Sequential Monte Carlo sampler for cosmological parameter estimation
• Conformal fixed point, Cosmological Constant and Quintessence
• Efficient acquisition rules for model-based approximate Bayesian computation
• ELFI: Engine for Likelihood-Free Inference
• Probabilistic learning of nonlinear dynamical systems using sequential Monte Carlo
• Accelerating pseudo-marginal Metropolis-Hastings by correlating auxiliary variables
• Classicality of primordial fluctuations and Primordial Black Holes
• Bayesian inference for stochastic differential equation mixed effects models of a tumor xenography study
• Approximate maximum likelihood estimation using data-cloning ABC
• Bayesian Inference for State Space Models using Block and Correlated Pseudo Marginal Methods
• Inference for the dark energy equation of state using Type IA supernova data
• The Block Pseudo-Marginal Sampler
• Bayesian epidemiological modeling over high-resolution network data
• A synthetic likelihood-based Laplace approximation for efficient design of biological processes
• Bayesian inference using synthetic likelihood: asymptotics and adjustments
• On the Conditional Distribution of the Multivariate $t$ Distribution
• Stratified sampling and bootstrapping for approximate Bayesian computation

Cited by in corpus (3)

• Generalized Bayesian Likelihood-Free Inference
• Guided sequential ABC schemes for intractable Bayesian models
• Stratified sampling and bootstrapping for approximate Bayesian computation