Rates of convergence of the Adaptive LASSO estimators to the Oracle distribution and higher order refinements by the bootstrap

Abstract

Zou [J. Amer. Statist. Assoc. 101 (2006) 1418-1429] proposed the Adaptive LASSO (ALASSO) method for simultaneous variable selection and estimation of the regression parameters, and established its oracle property. In this paper, we investigate the rate of convergence of the ALASSO estimator to the oracle distribution when the dimension of the regression parameters may grow to infinity with the sample size. It is shown that the rate critically depends on the choices of the penalty parameter and the initial estimator, among other factors, and that confidence intervals (CIs) based on the oracle limit law often have poor coverage accuracy. As an alternative, we consider the residual bootstrap method for the ALASSO estimators that has been recently shown to be consistent; cf. Chatterjee and Lahiri [J. Amer. Statist. Assoc. 106 (2011a) 608-625]. We show that the bootstrap applied to a suitable studentized version of the ALASSO estimator achieves second-order correctness, even when the dimension of the regression parameters is unbounded. Results from a moderately large simulation study show marked improvement in coverage accuracy for the bootstrap CIs over the oracle based CIs.

Published in at http://dx.doi.org/10.1214/13-AOS1106 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (7)

• Valid post-selection inference
• Asymptotic properties of bridge estimators in sparse high-dimensional regression models
• The sparsity and bias of the Lasso selection in high-dimensional linear regression
• Lasso-type recovery of sparse representations for high-dimensional data
• Sparsity oracle inequalities for the Lasso
• Sharp thresholds for high-dimensional and noisy recovery of sparsity
• Model-Consistent Sparse Estimation through the Bootstrap

Cited by in corpus (10)

• High-Dimensional Inference: Confidence Intervals, $p$-Values and R-Software hdi
• Confidence intervals for high-dimensional inverse covariance estimation
• Hierarchical inference for genome-wide association studies: a view on methodology with software
• Monte Carlo Simulation for Lasso-Type Problems by Estimator Augmentation
• Discussion: "A significance test for the lasso"
• Two-Stage Robust and Sparse Distributed Statistical Inference for Large-Scale Data
• Uncertainty Quantification for High Dimensional Sparse Nonparametric Additive Models
• Ultra high dimensional generalized additive model: Unified Theory and Methods
• ${\ell }_1$-penalized Multinomial Regression: Estimation, inference, and prediction, with an application to risk factor identification for different dementia subtypes
• Estimator Augmentation with Applications in High-Dimensional Group Inference