Fast Markov chain Monte Carlo sampling for sparse Bayesian inference in high-dimensional inverse problems using L1-type priors

Abstract

Sparsity has become a key concept for solving of high-dimensional inverse problems using variational regularization techniques. Recently, using similar sparsity-constraints in the Bayesian framework for inverse problems by encoding them in the prior distribution has attracted attention. Important questions about the relation between regularization theory and Bayesian inference still need to be addressed when using sparsity promoting inversion. A practical obstacle for these examinations is the lack of fast posterior sampling algorithms for sparse, high-dimensional Bayesian inversion: Accessing the full range of Bayesian inference methods requires being able to draw samples from the posterior probability distribution in a fast and efficient way. This is usually done using Markov chain Monte Carlo (MCMC) sampling algorithms. In this article, we develop and examine a new implementation of a single component Gibbs MCMC sampler for sparse priors relying on L1-norms. We demonstrate that the efficiency of our Gibbs sampler increases when the level of sparsity or the dimension of the unknowns is increased. This property is contrary to the properties of the most commonly applied Metropolis-Hastings (MH) sampling schemes: We demonstrate that the efficiency of MH schemes for L1-type priors dramatically decreases when the level of sparsity or the dimension of the unknowns is increased. Practically, Bayesian inversion for L1-type priors using MH samplers is not feasible at all. As this is commonly believed to be an intrinsic feature of MCMC sampling, the performance of our Gibbs sampler also challenges common beliefs about the applicability of sample based Bayesian inference.

33 pages, 14 figures

References in corpus (6)

• Optimal scaling for partially updating MCMC algorithms
• A Comparison of Methods for Computing Autocorrelation Time
• Discretization-invariant Bayesian inversion and Besov space priors
• Suppressing Random Walks in Markov Chain Monte Carlo Using Ordered Overrelaxation
• On optimal direction gibbs sampling
• Hierarchical models in statistical inverse problems and the Mumford--Shah functional

Cited by in corpus (11)

• Maximum-A-Posteriori Estimates in Linear Inverse Problems with Log-concave Priors are Proper Bayes Estimators
• Joint reconstruction strategy for structured illumination microscopy with unknown illuminations
• A hierarchical Bayesian perspective on majorization-minimization for non-convex sparse regression: application to M/EEG source imaging
• Solution Paths of Variational Regularization Methods for Inverse Problems
• Fast Gibbs sampling for high-dimensional Bayesian inversion
• On Compressed Sensing of Binary Signals for the Unsourced Random Access Channel
• Sparse online variational Bayesian regression
• Bayesian inverse problems with $l_1$ priors: a Randomize-then-Optimize approach
• Shape-constrained reconstruction in diffuse optical tomography by simulated annealing
• Bayesian Hierarchical Model of Total Variation Regularisation for Image Deblurring
• Optimization-Based MCMC Methods for Nonlinear Hierarchical Statistical Inverse Problems