On the adaptive elastic-net with a diverging number of parameters

Abstract

We consider the problem of model selection and estimation in situations where the number of parameters diverges with the sample size. When the dimension is high, an ideal method should have the oracle property [J. Amer. Statist. Assoc. 96 (2001) 1348--1360] and [Ann. Statist. 32 (2004) 928--961] which ensures the optimal large sample performance. Furthermore, the high-dimensionality often induces the collinearity problem, which should be properly handled by the ideal method. Many existing variable selection methods fail to achieve both goals simultaneously. In this paper, we propose the adaptive elastic-net that combines the strengths of the quadratic regularization and the adaptively weighted lasso shrinkage. Under weak regularity conditions, we establish the oracle property of the adaptive elastic-net. We show by simulations that the adaptive elastic-net deals with the collinearity problem better than the other oracle-like methods, thus enjoying much improved finite sample performance.

Published in at http://dx.doi.org/10.1214/08-AOS625 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (13)

• Rejoinder to "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Least Angle Regression
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Nonconcave penalized likelihood with a diverging number of parameters
• On the "degrees of freedom" of the lasso
• Statistical Challenges with High Dimensionality: Feature Selection in Knowledge Discovery

Cited by in corpus (73)

• Manifold Elastic Net: A Unified Framework for Sparse Dimension Reduction
• Penalized Composite Quasi-Likelihood for Ultrahigh-Dimensional Variable Selection
• Variable selection and regression analysis for graph-structured covariates with an application to genomics
• Endogeneity in high dimensions
• GEE analysis of clustered binary data with diverging number of covariates
• Variable selection in linear mixed effects models
• The sparse Laplacian shrinkage estimator for high-dimensional regression
• Consistent selection of tuning parameters via variable selection stability
• Least Squares Approximation for a Distributed System
• A majorization-minimization approach to variable selection using spike and slab priors
• The BerHu penalty and the grouped effect
• ICR: Iterative Convex Refinement for Sparse Signal Recovery Using Spike and Slab Priors
• Smooth Lasso Estimator for the Function-on-Function Linear Regression Model
• Convex and non-convex regularization methods for spatial point processes intensity estimation
• Adaptive Filter for Automatic Identification of Multiple Faults in a Noisy OTDR Profile
• Asymptotic properties for combined $L_1$ and concave regularization
• Model selection and structure specification in ultra-high dimensional generalised semi-varying coefficient models
• Robust Variable Selection and Estimation Via Adaptive Elastic Net S-Estimators for Linear Regression
• Regularization vs. Relaxation: A conic optimization perspective of statistical variable selection
• MM Algorithms for Minimizing Nonsmoothly Penalized Objective Functions
• Identifying Network Ties from Panel Data: Theory and an Application to Tax Competition
• Estimating Average Treatment Effects with a Double-Index Propensity Score
• Robust Periocular Recognition By Fusing Sparse Representations of Color and Geometry Information
• Scalable algorithms for semiparametric accelerated failure time models in high dimensions
• Multivariate functional group sparse regression: functional predictor selection
• A Hierarchical Integrative Group LASSO (HiGLASSO) Framework for Analyzing Environmental Mixtures
• FAStEN: An Efficient Adaptive Method for Feature Selection and Estimation in High-Dimensional Functional Regressions
• Adaptive group-regularized logistic elastic net regression
• Regression shrinkage and grouping of highly correlated predictors with HORSES
• Prior Adaptive Semi-supervised Learning with Application to EHR Phenotyping
• On Hodges' Superefficiency and Merits of Oracle Property in Model Selection
• Adaptive elastic-net selection in a quantile model with diverging number of variable groups
• Consistent Bayesian Information Criterion Based on a Mixture Prior for Possibly High-Dimensional Multivariate Linear Regression Models
• Binary Fused Compressive Sensing: 1-Bit Compressive Sensing meets Group Sparsity
• The Smooth-Lasso and other ${\ell }_1+{\ell }_2$-penalized methods
• Ridge Regression and Provable Deterministic Ridge Leverage Score Sampling
• Elastic Net based Feature Ranking and Selection
• Efficient Regularized Regression for Variable Selection with L0 Penalty
• Reconstructing large networks with time-varying interactions
• Learning LiNGAM based on data with more variables than observations
• Grouped Variable Selection via Nested Spike and Slab Priors
• Regularization for supervised learning via the "hubNet" procedure
• Endogenous Treatment Effect Estimation with some Invalid and Irrelevant Instruments
• Greedy Forward Regression for Variable Screening
• Sparse Density Estimation with Measurement Errors
• Variable selection in high-dimensional linear model with possibly asymmetric or heavy-tailed errors
• Adaptive elastic net and Separate Selection from Least Squares for ultra-high dimensional regression models
• Adaptive Elastic Net Method for Cox Model
• Variable Selection for Survival Data with A Class of Adaptive Elastic Net Techniques
• Penalty, Shrinkage, and Preliminary Test Estimators under Full Model Hypothesis
• Survival Modeling of Suicide Risk with Rare and Uncertain Diagnoses
• Spatial point processes intensity estimation with a diverging number of covariates
• Data Mining for Longitudinal Data under Multicollinearity and Time Dependence using Penalized Generalized Estimating Equations
• Sharp Convergence Rate and Support Consistency of Multiple Kernel Learning with Sparse and Dense Regularization
• A Weibull Mixture Cure Frailty Model for High-dimensional Covariates
• A Network-Guided Penalized Regression with Application to Proteomics Data
• A Multi-Omics Framework for Survival Mediation Analysis of High-Dimensional Proteogenomic Data
• Leveraging Two-Phase Data for Improved Prediction of Survival Outcomes with Application to Nasopharyngeal Cancer
• Generalized Co-sparse Factor Regression
• Regularization techniques for inhomogeneous (spatial) point processes intensity and conditional intensity estimation
• Pre-processing with Orthogonal Decompositions for High-dimensional Explanatory Variables
• yaglm: a Python package for fitting and tuning generalized linear models that supports structured, adaptive and non-convex penalties
• Sparse varying-coefficient functional linear model
• Integrated conditional moment test and beyond: when the number of covariates is divergent
• Spatial Variable Selection and An Application to Virginia Lyme Disease Emergence
• Health Analytics: a systematic review of approaches to detect phenotype cohorts using electronic health records
• Exponentially tilted likelihood inference on growing dimensional unconditional moment models
• Prediction Weighted Maximum Frequency Selection
• Fast and Scalable Lasso via Stochastic Frank-Wolfe Methods with a Convergence Guarantee
• Elastic Net Procedure for Partially Linear Models
• Sparse and Efficient Estimation for Partial Spline Models with Increasing Dimension
• Sparse Distance Weighted Discrimination
• Feature Screening via Distance Correlation Learning