Proximal MCMC for Bayesian Inference of Constrained and Regularized Estimation

Abstract

This paper advocates proximal Markov Chain Monte Carlo (ProxMCMC) as a flexible and general Bayesian inference framework for constrained or regularized estimation. Originally introduced in the Bayesian imaging literature, ProxMCMC employs the Moreau-Yosida envelope for a smooth approximation of the total-variation regularization term, fixes variance and regularization strength parameters as constants, and uses the Langevin algorithm for the posterior sampling. We extend ProxMCMC to be fully Bayesian by providing data-adaptive estimation of all parameters including the regularization strength parameter. More powerful sampling algorithms such as Hamiltonian Monte Carlo are employed to scale ProxMCMC to high-dimensional problems. Analogous to the proximal algorithms in optimization, ProxMCMC offers a versatile and modularized procedure for conducting statistical inference on constrained and regularized problems. The power of ProxMCMC is illustrated on various statistical estimation and machine learning tasks, the inference of which is traditionally considered difficult from both frequentist and Bayesian perspectives.

References in corpus (21)

• Discussion of "Least angle regression" by Efron et al
• Rejoinder to "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Least Angle Regression
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• On asymptotically optimal confidence regions and tests for high-dimensional models
• Confidence Intervals and Hypothesis Testing for High-Dimensional Regression
• Valid post-selection inference
• Tensor Regression with Applications in Neuroimaging Data Analysis
• Exact post-selection inference, with application to the lasso
• Sparsity information and regularization in the horseshoe and other shrinkage priors
• Regularized Matrix Regression
• Distance Majorization and Its Applications
• Generalized Linear Model Regression under Distance-to-set Penalties
• Feature Selection for Vertex Discriminant Analysis
• Distance-to-Set Priors and Constrained Bayesian Inference