Monte Carlo Simulation for Lasso-Type Problems by Estimator Augmentation
arXiv:1401.4425 · doi:10.1080/01621459.2014.946035
Abstract
Regularized linear regression under the penalty, such as the Lasso, has been shown to be effective in variable selection and sparse modeling. The sampling distribution of an -penalized estimator is hard to determine as the estimator is defined by an optimization problem that in general can only be solved numerically and many of its components may be exactly zero. Let be the subgradient of the norm of the coefficient vector evaluated at . We find that the joint sampling distribution of and , together called an augmented estimator, is much more tractable and has a closed-form density under a normal error distribution in both low-dimensional () and high-dimensional () settings. Given and the error variance , one may employ standard Monte Carlo methods, such as Markov chain Monte Carlo and importance sampling, to draw samples from the distribution of the augmented estimator and calculate expectations with respect to the sampling distribution of . We develop a few concrete Monte Carlo algorithms and demonstrate with numerical examples that our approach may offer huge advantages and great flexibility in studying sampling distributions in -penalized linear regression. We also establish nonasymptotic bounds on the difference between the true sampling distribution of and its estimator obtained by plugging in estimated parameters, which justifies the validity of Monte Carlo simulation from an estimated sampling distribution even when .
arXiv comment43 pages, 4 figures
References in corpus (24)
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Rejoinder to "Least angle regression" by Efron et al
- Least Angle Regression
- Discussion of "Least angle regression" by Efron et al
- Nearly unbiased variable selection under minimax concave penalty
- High-dimensional graphs and variable selection with the Lasso
- Simultaneous analysis of Lasso and Dantzig selector
- Pathwise coordinate optimization
- One-step sparse estimates in nonconcave penalized likelihood models
- Confidence Intervals and Hypothesis Testing for High-Dimensional Regression
- A significance test for the lasso
- Coordinate descent algorithms for lasso penalized regression
- The sparsity and bias of the Lasso selection in high-dimensional linear regression
- Lasso-type recovery of sparse representations for high-dimensional data
- Rates of convergence of the Adaptive LASSO estimators to the Oracle distribution and higher order refinements by the bootstrap
- Confidence Sets Based on Penalized Maximum Likelihood Estimators in Gaussian Regression
- Stability Selection
- Reversible jump Markov chain Monte Carlo and multi-model samplers
Cited by in corpus (8)
- Goodness of fit tests for high-dimensional linear models
- On the Distribution, Model Selection Properties and Uniqueness of the Lasso Estimator in Low and High Dimensions
- High-dimensional simultaneous inference with the bootstrap
- An MCMC-free approach to post-selective inference
- Constructing confidence sets after lasso selection by randomized estimator augmentation
- Uncertainty Quantification Under Group Sparsity
- Estimator Augmentation with Applications in High-Dimensional Group Inference
- Honest confidence sets for high-dimensional regression by projection and shrinkage