Tuning parameter selection in high dimensional penalized likelihood

Abstract

Determining how to appropriately select the tuning parameter is essential in penalized likelihood methods for high-dimensional data analysis. We examine this problem in the setting of penalized likelihood methods for generalized linear models, where the dimensionality of covariates p is allowed to increase exponentially with the sample size n. We propose to select the tuning parameter by optimizing the generalized information criterion (GIC) with an appropriate model complexity penalty. To ensure that we consistently identify the true model, a range for the model complexity penalty is identified in GIC. We find that this model complexity penalty should diverge at the rate of some power of $\log p$ depending on the tail probability behavior of the response variables. This reveals that using the AIC or BIC to select the tuning parameter may not be adequate for consistently identifying the true model. Based on our theoretical study, we propose a uniform choice of the model complexity penalty and show that the proposed approach consistently identifies the true model among candidate models with asymptotic probability one. We justify the performance of the proposed procedure by numerical simulations and a gene expression data analysis.

38 pages

References in corpus (4)

• Nearly unbiased variable selection under minimax concave penalty
• Coordinate descent algorithms for nonconvex penalized regression, with applications to biological feature selection
• Sure independence screening in generalized linear models with NP-dimensionality
• A unified approach to model selection and sparse recovery using regularized least squares

Cited by in corpus (21)

• Model Selection Principles in Misspecified Models
• Nonparametric independence screening and structure identification for ultra-high dimensional longitudinal data
• Model selection and structure specification in ultra-high dimensional generalised semi-varying coefficient models
• Forecasting: theory and practice
• Estimation and inference in generalized additive coefficient models for nonlinear interactions with high-dimensional covariates
• Ultrahigh-dimensional Robust and Efficient Sparse Regression using Non-Concave Penalized Density Power Divergence
• Variable Screening for High Dimensional Time Series
• High-dimensional variable selection via low-dimensional adaptive learning
• Analysis of overfitting in the regularized Cox model
• A Tensor-EM Method for Large-Scale Latent Class Analysis with Binary Responses
• High-dimensional Censored Regression via the Penalized Tobit Likelihood
• A study on tuning parameter selection for the high-dimensional lasso
• Parallel integrative learning for large-scale multi-response regression with incomplete outcomes
• Predictive Quantile Regression with Mixed Roots and Increasing Dimensions: The ALQR Approach
• A penalized inference approach to stochastic block modelling of community structure in the Italian Parliament
• Hierarchical correction of p-values via an ultrametric tree running Ornstein-Uhlenbeck process
• Two-Stage Robust and Sparse Distributed Statistical Inference for Large-Scale Data
• Ultra high dimensional generalized additive model: Unified Theory and Methods
• An Efficient Linearly Convergent Regularized Proximal Point Algorithm for Fused Multiple Graphical Lasso Problems
• High-Dimensional Expected Shortfall Regression
• Selection consistency of Lasso-based procedures for misspecified high-dimensional binary model and random regressors