Sparse linear discriminant analysis by thresholding for high dimensional data

Abstract

In many social, economical, biological and medical studies, one objective is to classify a subject into one of several classes based on a set of variables observed from the subject. Because the probability distribution of the variables is usually unknown, the rule of classification is constructed using a training sample. The well-known linear discriminant analysis (LDA) works well for the situation where the number of variables used for classification is much smaller than the training sample size. Because of the advance in technologies, modern statistical studies often face classification problems with the number of variables much larger than the sample size, and the LDA may perform poorly. We explore when and why the LDA has poor performance and propose a sparse LDA that is asymptotically optimal under some sparsity conditions on the unknown parameters. For illustration of application, we discuss an example of classifying human cancer into two classes of leukemia based on a set of 7,129 genes and a training sample of size 72. A simulation is also conducted to check the performance of the proposed method.

Published in at http://dx.doi.org/10.1214/10-AOS870 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (2)

• Covariance regularization by thresholding
• High-dimensional classification using features annealed independence rules

Cited by in corpus (14)

• Innovated interaction screening for high-dimensional nonlinear classification
• Optimal classification in sparse Gaussian graphic model
• Simultaneous sparse estimation of canonical vectors in the p>>N setting
• Regularized estimation of linear functionals of precision matrices for high-dimensional time series
• Non-parametric shrinkage mean estimation for quadratic loss functions with unknown covariance matrices
• Optimal feature selection for sparse linear discriminant analysis and its applications in gene expression data
• On the dimension effect of regularized linear discriminant analysis
• QUADRO: A supervised dimension reduction method via Rayleigh quotient optimization
• Optimal variable selection in multi-group sparse discriminant analysis
• Continuum directions for supervised dimension reduction
• On Two Simple and Effective Procedures for High Dimensional Classification of General Populations
• A Generative Approach to Joint Modeling of Quantitative and Qualitative Responses
• An Impossibility Result for High Dimensional Supervised Learning
• Dynamic Supervised Principal Component Analysis for Classification