Exploiting Multi-Core Architectures for Reduced-Variance Estimation with Intractable Likelihoods

Abstract

Many popular statistical models for complex phenomena are intractable, in the sense that the likelihood function cannot easily be evaluated. Bayesian estimation in this setting remains challenging, with a lack of computational methodology to fully exploit modern processing capabilities. In this paper we introduce novel control variates for intractable likelihoods that can dramatically reduce the Monte Carlo variance of Bayesian estimators. We prove that our control variates are well-defined and provide a positive variance reduction. Furthermore we show how to optimise these control variates for variance reduction. The methodology is highly parallel and offers a route to exploit multi-core processing architectures that complements recent research in this direction. Indeed, our work shows that it may not be necessary to parallelise the sampling process itself in order to harness the potential of massively multi-core architectures. Simulation results presented on the Ising model, exponential random graph models and non-linear stochastic differential equation models support our theoretical findings.

To appear in Bayesian Analysis, 2015

References in corpus (8)

• Statistical Modeling of Spatial Extremes
• MCMC for doubly-intractable distributions
• Austerity in MCMC Land: Cutting the Metropolis-Hastings Budget
• Zero-variance principle for Monte Carlo algorithms
• On Russian Roulette Estimates for Bayesian Inference with Doubly-Intractable Likelihoods
• Ergodicity of Approximate MCMC Chains with Applications to Large Data Sets
• The Controlled Thermodynamic Integral for Bayesian Model Comparison
• Noisy Monte Carlo: Convergence of Markov chains with approximate transition kernels