Ensemble Slice Sampling: Parallel, black-box and gradient-free inference for correlated & multimodal distributions

Abstract

Slice Sampling has emerged as a powerful Markov Chain Monte Carlo algorithm that adapts to the characteristics of the target distribution with minimal hand-tuning. However, Slice Sampling's performance is highly sensitive to the user-specified initial length scale hyperparameter and the method generally struggles with poorly scaled or strongly correlated distributions. This paper introduces Ensemble Slice Sampling (ESS), a new class of algorithms that bypasses such difficulties by adaptively tuning the initial length scale and utilising an ensemble of parallel walkers in order to efficiently handle strong correlations between parameters. These affine-invariant algorithms are trivial to construct, require no hand-tuning, and can easily be implemented in parallel computing environments. Empirical tests show that Ensemble Slice Sampling can improve efficiency by more than an order of magnitude compared to conventional MCMC methods on a broad range of highly correlated target distributions. In cases of strongly multimodal target distributions, Ensemble Slice Sampling can sample efficiently even in high dimensions. We argue that the parallel, black-box and gradient-free nature of the method renders it ideal for use in scientific fields such as physics, astrophysics and cosmology which are dominated by a wide variety of computationally expensive and non-differentiable models.

Published in Statistics & Computing; 18 pages, 11 figures, 2 tables; Code available https://github.com/minaskar/zeus

References in corpus (1)

• Ensemble Slice Sampling: Parallel, black-box and gradient-free inference for correlated & multimodal distributions

Cited by in corpus (11)

• A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges
• Unified galaxy power spectrum measurements from 6dFGS, BOSS, and eBOSS
• zeus: A Python implementation of Ensemble Slice Sampling for efficient Bayesian parameter inference
• The trouble beyond $H_0$ and the new cosmic triangles
• Ensemble Slice Sampling: Parallel, black-box and gradient-free inference for correlated & multimodal distributions
• Robust sampling for weak lensing and clustering analyses with the Dark Energy Survey
• Measuring line-of-sight shear with Einstein rings: a proof of concept
• A measurement of the Integrated Sachs-Wolfe Effect with the Rapid ASKAP Continuum Survey
• The effect of finite halo size on the clustering of neutral hydrogen
• A saddle-point finder and its application to the spin foam model
• A Latent Slice Sampling Algorithm