A General Framework for Bayes Structured Linear Models
arXiv:1506.02174
Abstract
High dimensional statistics deals with the challenge of extracting structured information from complex model settings. Compared with the growing number of frequentist methodologies, there are rather few theoretically optimal Bayes methods that can deal with very general high dimensional models. In contrast, Bayes methods have been extensively studied in various nonparametric settings and rate optimal posterior contraction results have been established. This paper provides a unified approach to both Bayes high dimensional statistics and Bayes nonparametrics in a general framework of structured linear models. With the proposed two-step model selection prior, we prove a general theorem of posterior contraction under an abstract setting. The main theorem can be used to derive new results on optimal posterior contraction under many complex model settings including stochastic block model, graphon estimation and dictionary learning. It can also be used to re-derive optimal posterior contraction for problems such as sparse linear regression and nonparametric aggregation, which improve upon previous Bayes results for these problems. The key of the success lies in the proposed two-step prior distribution. The prior on the parameters is an elliptical Laplace distribution that is capable to model signals with large magnitude, and the prior on the models involves an important correction factor that compensates the effect of the normalizing constant of the elliptical Laplace distribution.
References in corpus (11)
- Simultaneous analysis of Lasso and Dantzig selector
- Rates of contraction of posterior distributions based on Gaussian process priors
- Graph limits and exchangeable random graphs
- Rate-optimal graphon estimation
- Exponential Screening and optimal rates of sparse estimation
- Posterior convergence rates for estimating large precision matrices using graphical models
- A Clustering Approach to Learn Sparsely-Used Overcomplete Dictionaries
- Minimax Optimal Bayesian Aggregation
- Adaptive Minimax Estimation over Sparse -Hulls
- Volume Ratio, Sparsity, and Minimaxity under Unitarily Invariant Norms
- Optimal Bayesian estimation in stochastic block models
Cited by in corpus (13)
- Empirical Bayes posterior concentration in sparse high-dimensional linear models
- Bayesian Sparse Linear Regression with Unknown Symmetric Error
- Generalized linear models with low rank effects for network data
- Convergence Rates of Variational Posterior Distributions
- EM-Based Smooth Graphon Estimation Using Bayesian and Spline-Based Approaches
- General framework for projection structures
- Minimax Rates in Network Analysis: Graphon Estimation, Community Detection and Hypothesis Testing
- Oracle posterior contraction rates under hierarchical priors
- Optimal False Discovery Control of Minimax Estimator
- Posterior Asymptotic Normality for an Individual Coordinate in High-dimensional Linear Regression
- Bayesian Model Selection with Graph Structured Sparsity
- Rapid mixing of a Markov chain for an exponentially weighted aggregation estimator
- Shoiuld Humans Lie to Machines: The Incentive Compatibility of Lasso and General Weighted Lasso