Asymptotic properties of bridge estimators in sparse high-dimensional regression models

Abstract

We study the asymptotic properties of bridge estimators in sparse, high-dimensional, linear regression models when the number of covariates may increase to infinity with the sample size. We are particularly interested in the use of bridge estimators to distinguish between covariates whose coefficients are zero and covariates whose coefficients are nonzero. We show that under appropriate conditions, bridge estimators correctly select covariates with nonzero coefficients with probability converging to one and that the estimators of nonzero coefficients have the same asymptotic distribution that they would have if the zero coefficients were known in advance. Thus, bridge estimators have an oracle property in the sense of Fan and Li [J. Amer. Statist. Assoc. 96 (2001) 1348--1360] and Fan and Peng [Ann. Statist. 32 (2004) 928--961]. In general, the oracle property holds only if the number of covariates is smaller than the sample size. However, under a partial orthogonality condition in which the covariates of the zero coefficients are uncorrelated or weakly correlated with the covariates of nonzero coefficients, we show that marginal bridge estimators can correctly distinguish between covariates with nonzero and zero coefficients with probability converging to one even when the number of covariates is greater than the sample size.

Published in at http://dx.doi.org/10.1214/009053607000000875 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (4)

• Nonconcave penalized likelihood with a diverging number of parameters
• Variable selection using MM algorithms
• Boosting for high-dimensional linear models
• Sure Independence Screening for Ultra-High Dimensional Feature Space

Cited by in corpus (89)

• Nearly unbiased variable selection under minimax concave penalty
• Boosting Algorithms: Regularization, Prediction and Model Fitting
• Sure independence screening in generalized linear models with NP-dimensionality
• Least squares after model selection in high-dimensional sparse models
• Square-Root Lasso: Pivotal Recovery of Sparse Signals via Conic Programming
• Variable selection in nonparametric additive models
• Sparsistency and rates of convergence in large covariance matrix estimation
• Support union recovery in high-dimensional multivariate regression
• Robust rank correlation based screening
• A Selective Review of Group Selection in High-Dimensional Models
• Needles and Straw in a Haystack: Posterior concentration for possibly sparse sequences
• Statistical significance in high-dimensional linear models
• Penalized Composite Quasi-Likelihood for Ultrahigh-Dimensional Variable Selection
• SCAD-penalized regression in high-dimensional partially linear models
• Variable selection and regression analysis for graph-structured covariates with an application to genomics
• Coauthorship and Citation Networks for Statisticians
• Calibrating nonconvex penalized regression in ultra-high dimension
• Lasso and probabilistic inequalities for multivariate point processes
• The fused Kolmogorov filter: A nonparametric model-free screening method
• Endogeneity in high dimensions
• The Lasso for High-Dimensional Regression with a Possible Change-Point
• GEE analysis of clustered binary data with diverging number of covariates
• Consistent group selection in high-dimensional linear regression
• Rates of convergence of the Adaptive LASSO estimators to the Oracle distribution and higher order refinements by the bootstrap
• Marginal empirical likelihood and sure independence feature screening
• Discussion: One-step sparse estimates in nonconcave penalized likelihood models
• Improved variable selection with Forward-Lasso adaptive shrinkage
• Independence test for high dimensional data based on regularized canonical correlation coefficients
• Asymptotic oracle properties of SCAD-penalized least squares estimators
• Variable selection in functional regression models: a review
• A Selective Overview of Variable Selection in High Dimensional Feature Space (Invited Review Article)
• Model selection and structure specification in ultra-high dimensional generalised semi-varying coefficient models
• High-dimensional influence measure
• Regularization vs. Relaxation: A conic optimization perspective of statistical variable selection
• Adaptive LASSO estimation for functional hidden dynamic geostatistical model
• Adaptive sparse group LASSO in quantile regression
• A general theory of regression adjustment for covariate-adaptive randomization: OLS, Lasso, and beyond
• A Bootstrap Lasso + Partial Ridge Method to Construct Confidence Intervals for Parameters in High-dimensional Sparse Linear Models
• Selection of tuning parameters in bridge regression models via Bayesian information criterion
• Sure Independence Screening for Ultra-High Dimensional Feature Space
• Which bridge estimator is optimal for variable selection?
• A Model-free Variable Screening Method Based on Leverage Score
• Alternating Direction Method of Multipliers for A Class of Nonconvex and Nonsmooth Problems with Applications to Background/Foreground Extraction
• Joint estimation of sparse multivariate regression and conditional graphical models
• Proximal Gradient Method with Extrapolation and Line Search for a Class of Nonconvex and Nonsmooth Problems
• Some Two-Step Procedures for Variable Selection in High-Dimensional Linear Regression
• ExSIS: Extended Sure Independence Screening for Ultrahigh-dimensional Linear Models
• Functional Group Bridge for Simultaneous Regression and Support Estimation
• Uniform Inference in High-dimensional Dynamic Panel Data Models
• Overcoming The Limitations of Phase Transition by Higher Order Analysis of Regularization Techniques
• Asymptotic Properties of Lasso+mLS and Lasso+Ridge in Sparse High-dimensional Linear Regression
• Nonparametric Independence Screening in Sparse Ultra-High Dimensional Additive Models
• Sequential Lasso for feature selection with ultra-high dimensional feature space
• Quantum-Inspired Hamiltonian Monte Carlo for Bayesian Sampling
• M-estimation in high-dimensional linear model
• Shrinkage Estimation and Selection for Multiple Functional Regression
• Adaptive post-Dantzig estimation and prediction for non-sparse "large $p$ and small $n$" models
• Complexity of Unconstrained L_2-L_p Minimization
• Regularized deep learning with nonconvex penalties
• Simulation-based Regularized Logistic Regression
• Fast and efficient algorithms for sparse semiparametric bi-functional regression
• Penalty methods for a class of non-Lipschitz optimization problems
• Dynamic Large Spatial Covariance Matrix Estimation in Application to Semiparametric Model Construction via Variable Clustering: the SCE approach
• An analysis of the SPARSEVA estimate for the finite sample data case
• Generalization error minimization: a new approach to model evaluation and selection with an application to penalized regression
• Bayesian iterative screening in ultra-high dimensional linear regressions
• A novel nonconvex, smooth-at-origin penalty for statistical learning
• Weak Signal Identification and Inference in Penalized Model Selection
• Estimation and inference for high-dimensional non-sparse models
• Data augmentation for non-Gaussian regression models using variance-mean mixtures
• Adaptive elastic net and Separate Selection from Least Squares for ultra-high dimensional regression models
• High-Dimensional Semiparametric Selection Models: Estimation Theory with an Application to the Retail Gasoline Market
• Scalable Algorithms for the Sparse Ridge Regression
• Independence Test for High Dimensional Random Vectors
• Fused Mean-variance Filter for Feature Screening
• High-dimensional Ordinary Least-squares Projection for Screening Variables
• On the use of cross-validation for the calibration of the adaptive lasso
• Bayesian $L_{\frac{1}{2}}$ regression
• Variable Screenings in Binary Response Regressions with Multivariate Normal Predictors
• Projection Neural Network for a Class of Sparse Regression Problems with Cardinality Penalty
• Parameter estimation of high-dimensional linear differential equations
• The Ridge Path Estimator for Linear Instrumental Variables
• Testing (Infinitely) Many Zero Restrictions
• High-Dimensional Bayesian Inference in Nonparametric Additive Models
• The ET Interview: Professor Joel L. Horowitz
• Unified Smoothing Approach for Best Hyperparameter Selection Problem Using a Bilevel Optimization Strategy
• Sparse and Efficient Estimation for Partial Spline Models with Increasing Dimension
• Pre-processing with Orthogonal Decompositions for High-dimensional Explanatory Variables
• Selective Inference via Marginal Screening for High Dimensional Classification