PCA consistency in high dimension, low sample size context

Abstract

Principal Component Analysis (PCA) is an important tool of dimension reduction especially when the dimension (or the number of variables) is very high. Asymptotic studies where the sample size is fixed, and the dimension grows [i.e., High Dimension, Low Sample Size (HDLSS)] are becoming increasingly relevant. We investigate the asymptotic behavior of the Principal Component (PC) directions. HDLSS asymptotics are used to study consistency, strong inconsistency and subspace consistency. We show that if the first few eigenvalues of a population covariance matrix are large enough compared to the others, then the corresponding estimated PC directions are consistent or converge to the appropriate subspace (subspace consistency) and most other PC directions are strongly inconsistent. Broad sets of sufficient conditions for each of these cases are specified and the main theorem gives a catalogue of possible combinations. In preparation for these results, we show that the geometric representation of HDLSS data holds under general conditions, which includes a $\rho$-mixing condition and a broad range of sphericity measures of the covariance matrix.

Published in at http://dx.doi.org/10.1214/09-AOS709 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (2)

• Basic Properties of Strong Mixing Conditions. A Survey and Some Open Questions
• Sparse Principal Components Analysis

Cited by in corpus (64)

• Sparse principal component analysis and iterative thresholding
• Sparse PCA: Optimal rates and adaptive estimation
• Projected principal component analysis in factor models
• Convergence and prediction of principal component scores in high-dimensional settings
• On perfect clustering of high dimension, low sample size data
• Statistical analysis of latent generalized correlation matrix estimation in transelliptical distribution
• Sparse and Functional Principal Components Analysis
• Asymptotics of Empirical Eigen-structure for Ultra-high Dimensional Spiked Covariance Model
• Distributed Estimation of Principal Eigenspaces
• Surprising Asymptotic Conical Structure in Critical Sample Eigen-Directions
• Regularized Tensor Factorizations and Higher-Order Principal Components Analysis
• Two-sample tests for high-dimension, strongly spiked eigenvalue models
• Asymptotic properties of Principal Component Analysis and shrinkage-bias adjustment under the Generalized Spiked Population model
• Sequential testing for structural stability in approximate factor models
• Penalized Orthogonal Iteration for Sparse Estimation of Generalized Eigenvalue Problem
• Limiting Laws for Divergent Spiked Eigenvalues and Largest Non-spiked Eigenvalue of Sample Covariance Matrices
• An algorithm for deciding the number of clusters and validation using simulated data with application to exploring crop population structure
• Classification Using Global and Local Mahalanobis Distances
• When and why are principal component scores a good tool for visualizing high-dimensional data?
• Estimation of functionals of sparse covariance matrices
• On the number of principal components in high dimensions
• On high-dimensional modifications of some graph-based two-sample tests
• Distance-based and RKHS-based Dependence Metrics in High Dimension
• Unbounded Largest Eigenvalue of Large Sample Covariance Matrices: Asymptotics, Fluctuations and Applications
• An Overview on the Estimation of Large Covariance and Precision Matrices
• Multi-scale Classification using Localized Spatial Depth
• Influential Feature PCA for high dimensional clustering
• Evaluation of the number of clusters in a data set using $p$-values from Multiple Tests of Hypotheses
• Optimal Estimation and Rank Detection for Sparse Spiked Covariance Matrices
• Risk of the Least Squares Minimum Norm Estimator under the Spike Covariance Model
• Continuum directions for supervised dimension reduction
• Generalized Four Moment Theorem and an Application to CLT for Spiked Eigenvalues of Large-dimensional Covariance Matrices
• Principal component analysis based clustering for high-dimension, low-sample-size data
• High Dimensional Principal Component Scores and Data Visualization
• Mean-field theory of Bayesian clustering
• Data-Space Validation of High-Dimensional Models by Comparing Sample Quantiles
• Feature space reduction method for ultrahigh-dimensional, multiclass data: Random forest-based multiround screening (RFMS)
• Joint CLT for top eigenvalues of sample covariance matrices of separable high dimensional long memory processes
• Empirical Bayes PCA in high dimensions
• An ${\ell }_p$ theory of PCA and spectral clustering
• Extending the Davis-Kahan theorem for comparing eigenvectors of two symmetric matrices I: Theory
• Sparse Sliced Inverse Regression Via Lasso
• The limits of the sample spiked eigenvalues for a high-dimensional generalized Fisher matrix and its applications
• "Virus hunting" using radial distance weighted discrimination
• Nonsparse learning with latent variables
• Testing for subsphericity when $n$ and $p$ are of different asymptotic order
• Generalized Four Moment Theorem with an application to the CLT for the spiked eigenvalues of high-dimensional general Fisher-matrices
• Theory of high-dimensional outliers
• On High Dimensional Behaviour of Some Two-Sample Tests Based on Ball Divergence
• Inconsistency of diagonal scaling under high-dimensional limit: a replica approach
• Adaptive Reference-Guided Estimation of Principal Component Subspace in High Dimensions
• Model-based clustering in very high dimensions via adaptive projections
• Significance Analysis of High-Dimensional, Low-Sample Size Partially Labeled Data
• Computational Anatomy for Multi-Organ Analysis in Medical Imaging: A Review
• Low-Rank Principal Eigenmatrix Analysis
• High-Dimension, Low Sample Size Asymptotics of Canonical Correlation Analysis
• Homogeneity Tests of Covariance and Change-Points Identification for High-Dimensional Functional Data
• Interpretable Network Representation Learning with Principal Component Analysis
• Steerable $e$PCA: Rotationally Invariant Exponential Family PCA
• Partial Least Squares for Functional Joint Models
• Asymptotic properties of the first principal component and equality tests of covariance matrices in high-dimension, low-sample-size context
• Extending the Davis-Kahan theorem for comparing eigenvectors of two symmetric matrices II: Computation and Applications
• Regularized Partial Least Squares with an Application to NMR Spectroscopy
• Multidimensional molecular changes-environment interaction analysis for disease outcomes