On the utility of graphics cards to perform massively parallel simulation of advanced Monte Carlo methods

Abstract

We present a case-study on the utility of graphics cards to perform massively parallel simulation of advanced Monte Carlo methods. Graphics cards, containing multiple Graphics Processing Units (GPUs), are self-contained parallel computational devices that can be housed in conventional desktop and laptop computers. For certain classes of Monte Carlo algorithms they offer massively parallel simulation, with the added advantage over conventional distributed multi-core processors that they are cheap, easily accessible, easy to maintain, easy to code, dedicated local devices with low power consumption. On a canonical set of stochastic simulation examples including population-based Markov chain Monte Carlo methods and Sequential Monte Carlo methods, we find speedups from 35 to 500 fold over conventional single-threaded computer code. Our findings suggest that GPUs have the potential to facilitate the growth of statistical modelling into complex data rich domains through the availability of cheap and accessible many-core computation. We believe the speedup we observe should motivate wider use of parallelizable simulation methods and greater methodological attention to their design.

Expansion of details; a condensed version has been submitted for publication

References in corpus (1)

• The incomplete beta function law for parallel tempering sampling of classical canonical systems

Cited by in corpus (26)

• Adaptive tempered reversible jump algorithm for Bayesian curve fitting
• On the role of interaction in sequential Monte Carlo algorithms
• Delayed acceptance particle MCMC for exact inference in stochastic kinetic models
• Leave Pima Indians alone: binary regression as a benchmark for Bayesian computation
• Variance bounding and geometric ergodicity of Markov chain Monte Carlo kernels for approximate Bayesian computation
• QInfer: Statistical inference software for quantum applications
• A self-organizing state-space-model approach for parameter estimation in Hodgkin-Huxley-type models of single neurons
• Path storage in the particle filter
• GPU-accelerated Gibbs sampling: a case study of the Horseshoe Probit model
• Graphics Processing Units and High-Dimensional Optimization
• Importance sampling type estimators based on approximate marginal MCMC
• Rapid Bayesian inference for expensive stochastic models
• Approximate Smoothing and Parameter Estimation in High-Dimensional State-Space Models
• Dynamic filtering of static dipoles in magnetoencephalography
• How to identify earth pressures on in-service tunnel linings: Insights from Bayesian inversion to address non-uniqueness
• Increasing the efficiency of Sequential Monte Carlo samplers through the use of approximately optimal L-kernels
• Accelerated dimension-independent adaptive Metropolis
• Static and dynamic SABR stochastic volatility models: calibration and option pricing using GPUs
• Multifidelity multilevel Monte Carlo to accelerate approximate Bayesian parameter inference for partially observed stochastic processes
• Importance sampling correction versus standard averages of reversible MCMCs in terms of the asymptotic variance
• Vector operations for accelerating expensive Bayesian computations -- a tutorial guide
• The Time Machine: A Simulation Approach for Stochastic Trees
• Vectorized and Parallel Particle Filter SMC Parameter Estimation for Stiff ODEs
• Bayesian non-parametric inference for $\Lambda$-coalescents: consistency and a parametric method
• Inference and rare event simulation for stopped Markov processes via reverse-time sequential Monte Carlo
• Anytime Monte Carlo