A General Framework for Bayes Structured Linear Models

Abstract

High dimensional statistics deals with the challenge of extracting structured information from complex model settings. Compared with the growing number of frequentist methodologies, there are rather few theoretically optimal Bayes methods that can deal with very general high dimensional models. In contrast, Bayes methods have been extensively studied in various nonparametric settings and rate optimal posterior contraction results have been established. This paper provides a unified approach to both Bayes high dimensional statistics and Bayes nonparametrics in a general framework of structured linear models. With the proposed two-step model selection prior, we prove a general theorem of posterior contraction under an abstract setting. The main theorem can be used to derive new results on optimal posterior contraction under many complex model settings including stochastic block model, graphon estimation and dictionary learning. It can also be used to re-derive optimal posterior contraction for problems such as sparse linear regression and nonparametric aggregation, which improve upon previous Bayes results for these problems. The key of the success lies in the proposed two-step prior distribution. The prior on the parameters is an elliptical Laplace distribution that is capable to model signals with large magnitude, and the prior on the models involves an important correction factor that compensates the effect of the normalizing constant of the elliptical Laplace distribution.

References in corpus (4)

• Rates of contraction of posterior distributions based on Gaussian process priors
• Graph limits and exchangeable random graphs
• Minimax Optimal Bayesian Aggregation
• Optimal Bayesian estimation in stochastic block models

Cited by in corpus (12)

• Bayesian Sparse Linear Regression with Unknown Symmetric Error
• Generalized linear models with low rank effects for network data
• Convergence Rates of Variational Posterior Distributions
• EM-Based Smooth Graphon Estimation Using Bayesian and Spline-Based Approaches
• Minimax Rates in Network Analysis: Graphon Estimation, Community Detection and Hypothesis Testing
• General framework for projection structures
• Oracle posterior contraction rates under hierarchical priors
• Optimal False Discovery Control of Minimax Estimator
• Posterior Asymptotic Normality for an Individual Coordinate in High-dimensional Linear Regression
• Rapid mixing of a Markov chain for an exponentially weighted aggregation estimator
• Bayesian Model Selection with Graph Structured Sparsity
• Shoiuld Humans Lie to Machines: The Incentive Compatibility of Lasso and General Weighted Lasso