Sure independence screening in generalized linear models with NP-dimensionality

Abstract

Ultrahigh-dimensional variable selection plays an increasingly important role in contemporary scientific discoveries and statistical research. Among others, Fan and Lv [J. R. Stat. Soc. Ser. B Stat. Methodol. 70 (2008) 849-911] propose an independent screening framework by ranking the marginal correlations. They showed that the correlation ranking procedure possesses a sure independence screening property within the context of the linear model with Gaussian covariates and responses. In this paper, we propose a more general version of the independent learning with ranking the maximum marginal likelihood estimates or the maximum marginal likelihood itself in generalized linear models. We show that the proposed methods, with Fan and Lv [J. R. Stat. Soc. Ser. B Stat. Methodol. 70 (2008) 849-911] as a very special case, also possess the sure screening property with vanishing false selection rate. The conditions under which the independence learning possesses a sure screening is surprisingly simple. This justifies the applicability of such a simple method in a wide spectrum. We quantify explicitly the extent to which the dimensionality can be reduced by independence screening, which depends on the interactions of the covariance matrix of covariates and true parameters. Simulation studies are used to illustrate the utility of the proposed approaches. In addition, we establish an exponential inequality for the quasi-maximum likelihood estimator which is useful for high-dimensional statistical learning.

Published in at http://dx.doi.org/10.1214/10-AOS798 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (3)

• Asymptotic properties of bridge estimators in sparse high-dimensional regression models
• High-dimensional classification using features annealed independence rules
• High-dimensional generalized linear models and the lasso

Cited by in corpus (88)

• Challenges of Big Data Analysis
• Strong oracle optimality of folded concave penalized estimation
• Robust rank correlation based screening
• Tuning parameter selection in high dimensional penalized likelihood
• Quantile-adaptive model-free variable screening for high-dimensional heterogeneous data
• Penalized Composite Quasi-Likelihood for Ultrahigh-Dimensional Variable Selection
• The fused Kolmogorov filter: A nonparametric model-free screening method
• Exact Post Model Selection Inference for Marginal Screening
• Model-free Feature Screening and FDR Control with Knockoff Features
• Marginal empirical likelihood and sure independence feature screening
• Distributed Estimation and Inference with Statistical Guarantees
• Sparse Regression: Scalable algorithms and empirical performance
• Nonparametric independence screening and structure identification for ultra-high dimensional longitudinal data
• Linear Hypothesis Testing in Dense High-Dimensional Linear Models
• High-Dimensional Multivariate Posterior Consistency Under Global-Local Shrinkage Priors
• Covariate assisted screening and estimation
• A Selective Overview of Variable Selection in High Dimensional Feature Space (Invited Review Article)
• Sparse Classification: a scalable discrete optimization perspective
• Local independence feature screening for nonparametric and semiparametric models by marginal empirical likelihood
• Model selection and structure specification in ultra-high dimensional generalised semi-varying coefficient models
• High-dimensional influence measure
• Multiclass Sparse Discriminant Analysis
• An Overview of Healthcare Data Analytics With Applications to the COVID-19 Pandemic
• Interaction Pursuit with Feature Screening and Selection
• Sure Screening for Gaussian Graphical Models
• Control of the False Discovery Rate Under Arbitrary Covariance Dependence
• Variable Screening for High Dimensional Time Series
• Sparse Additive Functional and Kernel CCA
• The Backbone Method for Ultra-High Dimensional Sparse Machine Learning
• Two-stage Sampling, Prediction and Adaptive Regression via Correlation Screening (SPARCS)
• A Likelihood Ratio Framework for High Dimensional Semiparametric Regression
• ExSIS: Extended Sure Independence Screening for Ultrahigh-dimensional Linear Models
• Feature selection when there are many influential features
• The Kendall Interaction Filter for Variable Interaction Screening in Ultra High Dimensional Classification Problems
• Estimation and Inference for High Dimensional Generalized Linear Models: A Splitting and Smoothing Approach
• Nonparametric Independence Screening via Favored Smoothing Bandwidth
• Regularization Approach for Network Modeling of German Power Derivative Market
• Nonparametric Screening under Conditional Strictly Convex Loss for Ultrahigh Dimensional Sparse Data
• Nonparametric Independence Screening in Sparse Ultra-High Dimensional Additive Models
• Likelihood Ratio Test in Multivariate Linear Regression: from Low to High Dimension
• (A) Data in the Life: Authorship Attribution of Lennon-McCartney Songs
• Modular Bayes screening for high-dimensional predictors
• Comparisons of penalized least squares methods by simulations
• Regularization after retention in ultrahigh dimensional linear regression models
• High-Dimensional Screening Using Multiple Grouping of Variables
• Covariate Assisted Variable Ranking
• The cumulative Kolmogorov filter for model-free screening in ultrahigh dimensional data
• Ultra high dimensional generalized additive model: Unified Theory and Methods
• An adaptive shortest-solution guided decimation approach to sparse high-dimensional linear regression
• Markov Neighborhood Regression for High-Dimensional Inference
• Copula-based Partial Correlation Screening: a Joint and Robust Approach
• Strong Sure Screening of Ultra-high Dimensional Data with Interaction Effects
• Selection Consistency of EBIC for GLIM with Non-canonical Links and Diverging Number of Parameters
• Bayesian Ultrahigh-Dimensional Screening Via MCMC
• Variance Estimation Using Refitted Cross-validation in Ultrahigh Dimensional Regression
• Discrete-time Competing-Risks Regression with or without Penalization
• Greedy Forward Regression for Variable Screening
• Computationally efficient univariate filtering for massive data
• Bayesian iterative screening in ultra-high dimensional linear regressions
• Variable screening with multiple studies
• Partial Distance Correlation Screening for High Dimensional Time Series
• Model-Free Conditional Feature Screening with Exposure Variables
• Communication-Efficient Distributed Estimator for Generalized Linear Models with a Diverging Number of Covariates
• Discussion: "A significance test for the lasso"
• A flexible model-free prediction-based framework for feature ranking
• Local Linear Regression on Manifolds and its Geometric Interpretation
• Independent screening for single-index hazard rate models with ultra-high dimensional features
• Estimating False Discovery Proportion Under Arbitrary Covariance Dependence
• Nonparametric Variable Screening with Optimal Decision Stumps
• Variable Screenings in Binary Response Regressions with Multivariate Normal Predictors
• Forward variable selection for sparse ultra-high dimensional varying coefficient models
• An Efficient Post-Selection Inference on High-Order Interaction Models
• Fused Mean-variance Filter for Feature Screening
• Ultra High Dimensional Change Point Detection
• High-Dimensional Bayesian Inference in Nonparametric Additive Models
• High-Dimensional Inference Based on the Leave-One-Covariate-Out LASSO Path
• Linear screening for high-dimensional computer experiments
• Ranking genetic factors related to age-related maculardegeneration by variable selection confidence sets
• Feature Screening via Distance Correlation Learning
• Conditional variable screening via ordinary least squares projection
• Variable Selection for Additive Partial Linear Quantile Regression with Missing Covariates
• Marginal and Interactive Feature Screening of Ultra-high Dimensional Feature Spaces with Multivariate Response
• Data-Driven Random Projection and Screening for High-Dimensional Generalized Linear Models
• Iterated Feature Screening based on Distance Correlation for Ultrahigh-Dimensional Censored Data with Covariates Measurement Error
• Selective Inference via Marginal Screening for High Dimensional Classification
• Model-Free Sure Screening via Maximum Correlation
• High-dimensional Ordinary Least-squares Projection for Screening Variables
• A sure independence screening procedure for ultra-high dimensional partially linear additive models