Innovated interaction screening for high-dimensional nonlinear classification

Abstract

This paper is concerned with the problems of interaction screening and nonlinear classification in a high-dimensional setting. We propose a two-step procedure, IIS-SQDA, where in the first step an innovated interaction screening (IIS) approach based on transforming the original $p$-dimensional feature vector is proposed, and in the second step a sparse quadratic discriminant analysis (SQDA) is proposed for further selecting important interactions and main effects and simultaneously conducting classification. Our IIS approach screens important interactions by examining only $p$ features instead of all two-way interactions of order $O(p^2)$. Our theory shows that the proposed method enjoys sure screening property in interaction selection in the high-dimensional setting of $p$ growing exponentially with the sample size. In the selection and classification step, we establish a sparse inequality on the estimated coefficient vector for QDA and prove that the classification error of our procedure can be upper-bounded by the oracle classification error plus some smaller order term. Extensive simulation studies and real data analysis show that our proposal compares favorably with existing methods in interaction selection and high-dimensional classification.

Published at http://dx.doi.org/10.1214/14-AOS1308 in the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (14)

• Nearly unbiased variable selection under minimax concave penalty
• Simultaneous analysis of Lasso and Dantzig selector
• Regularized estimation of large covariance matrices
• Sparse permutation invariant covariance estimation
• High-dimensional classification using features annealed independence rules
• High-dimensional generalized linear models and the lasso
• A unified approach to model selection and sparse recovery using regularized least squares
• Sparse linear discriminant analysis by thresholding for high dimensional data
• Innovated higher criticism for detecting sparse signals in correlated noise
• Honest variable selection in linear and logistic regression models via ${\ell }_1$ and ${\ell }_1+{\ell }_2$ penalization
• Variable selection for general index models via sliced inverse regression
• Asymptotic equivalence of regularization methods in thresholded parameter space
• Optimal classification in sparse Gaussian graphic model
• Structures and Assumptions: Strategies to Harness Gene $\times$ Gene and Gene $\times$ Environment Interactions in GWAS

Cited by in corpus (8)

• A Direct Approach for Sparse Quadratic Discriminant Analysis
• Intentional Control of Type I Error over Unconscious Data Distortion: a Neyman-Pearson Approach to Text Classification
• The Kendall Interaction Filter for Variable Interaction Screening in Ultra High Dimensional Classification Problems
• RANK: Large-Scale Inference with Graphical Nonlinear Knockoffs
• RaSE: Random Subspace Ensemble Classification
• Interaction pursuit in high-dimensional multi-response regression via distance correlation
• Penalized Interaction Estimation for Ultrahigh Dimensional Quadratic Regression
• An Integrated Inverse Space Sparse Representation Framework for Tumor Classification