A two-sample test for high-dimensional data with applications to gene-set testing

Abstract

We propose a two-sample test for the means of high-dimensional data when the data dimension is much larger than the sample size. Hotelling's classical $T^2$ test does not work for this "large $p$, small $n$" situation. The proposed test does not require explicit conditions in the relationship between the data dimension and sample size. This offers much flexibility in analyzing high-dimensional data. An application of the proposed test is in testing significance for sets of genes which we demonstrate in an empirical study on a leukemia data set.

Published in at http://dx.doi.org/10.1214/09-AOS716 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (6)

• On testing the significance of sets of genes
• On Orthogonal and Symplectic Matrix Ensembles
• A two-sample test for high-dimensional data with applications to gene-set testing
• Random-set methods identify distinct aspects of the enrichment signal in gene-set analysis
• Multiple tests of association with biological annotation metadata
• To how many simultaneous hypothesis tests can normal, Student's t or bootstrap calibration be applied?

Cited by in corpus (92)

• Selective review of offline change point detection methods
• A two-sample test for high-dimensional data with applications to gene-set testing
• Two sample tests for high-dimensional covariance matrices
• Uniform change point tests in high dimension
• Gains in Power from Structured Two-Sample Tests of Means on Graphs
• More power via graph-structured tests for differential expression of gene networks
• Nonparametric Stein-type Shrinkage Covariance Matrix Estimators in High-Dimensional Settings
• On the sphericity test with large-dimensional observations
• Comparing large covariance matrices under weak conditions on the dependence structure and its application to gene clustering
• Tests alternative to higher criticism for high-dimensional means under sparsity and column-wise dependence
• A high-dimensional two-sample test for the mean using random subspaces
• Simulation-Based Hypothesis Testing of High Dimensional Means Under Covariance Heterogeneity
• A More Powerful Two-Sample Test in High Dimensions using Random Projection
• An adaptable generalization of Hotelling's $T^2$ test in high dimension
• A review of 20 years of naive tests of significance for high-dimensional mean vectors and covariance matrices
• Two-sample Hypothesis Testing for Inhomogeneous Random Graphs
• Bootstrapping High Dimensional Time Series
• Central limit theorems for high dimensional dependent data
• Cramér-type moderate deviations for Studentized two-sample $U$-statistics with applications
• A Cramér moderate deviation theorem for Hotelling's $T^2$-statistic with applications to global tests
• Two-sample tests for high-dimension, strongly spiked eigenvalue models
• Kernel Two-Sample Tests in High Dimension: Interplay Between Moment Discrepancy and Dimension-and-Sample Orders
• Tests for covariance matrix with fixed or divergent dimension
• Estimation of functionals of sparse covariance matrices
• Two-Sample Testing in High-Dimensional Models
• Classification accuracy as a proxy for two sample testing
• Testing the independence of two random vectors where only one dimension is large
• MMD Aggregated Two-Sample Test
• Multiple-Splitting Projection Test for High-Dimensional Mean Vectors
• A new interpoint distance-based clustering algorithm using kernel density estimation
• Testing the Mean Matrix in High-Dimensional Transposable Data
• Robust Multivariate Nonparametric Tests via Projection-Averaging
• $L^2$ Asymptotics for High-Dimensional Data
• Direction-Projection-Permutation for High Dimensional Hypothesis Tests
• Change Point Detection in the Mean of High-Dimensional Time Series Data under Dependence
• On the High-dimensional Power of Linear-time Kernel Two-Sample Testing under Mean-difference Alternatives
• Dating the Break in High-dimensional Data
• Test for a Mean Vector with Fixed or Divergent Dimension
• Graph-Based Two-Sample Tests for Data with Repeated Observations
• Manifold Asymptotics of Quadratic-Form-Based Inference in Repeated Measures Designs
• Testing High Dimensional Mean Under Sparsity
• Asymptotically Independent U-Statistics in High-Dimensional Testing
• Joint mean and covariance estimation with unreplicated matrix-variate data
• Adaptive post-Dantzig estimation and prediction for non-sparse "large $p$ and small $n$" models
• CLT for large dimensional general Fisher matrices and its applications in high-dimensional data analysis
• Covariate Assisted Variable Ranking
• Online Change-Point Detection in High-Dimensional Covariance Structure with Application to Dynamic Networks
• A Wilcoxon-Mann-Whitney type test for infinite dimensional data
• A Neighborhood-Assisted Hotelling's $T^2$ Test for High-Dimensional Means
• An approximate randomization test for high-dimensional two-sample Behrens-Fisher problem under arbitrary covariances
• Adaptive Inference for Change Points in High-Dimensional Data
• Graph Estimation for Matrix-variate Gaussian Data
• Power Enhancement in High Dimensional Cross-Sectional Tests
• Test for Temporal Homogeneity of Means in High-dimensional Longitudinal Data
• Distribution and correlation free two-sample test of high-dimensional means
• Distance-based classifier by data transformation for high-dimension, strongly spiked eigenvalue models
• Optimal estimation of functionals of high-dimensional mean and covariance matrix
• Spectral Analysis of Gram Matrices with Missing at Random Observations: Convergence, Central Limit Theorems, and Applications in Statistical Inference
• High-dimensional general linear hypothesis tests via non-linear spectral shrinkage
• Mean Test with Fewer Observation than Dimension and Ratio Unbiased Estimator for Correlation Matrix
• On High Dimensional Behaviour of Some Two-Sample Tests Based on Ball Divergence
• Test for mean matrix in GMANOVA model under heteroscedasticity and non-normality for high-dimensional data
• Interpoint Distance Based Two Sample Tests in High Dimension
• Hypothesis Testing of One-Sample Mean Vector in Distributed Frameworks
• A Change-Point Based Control Chart for Detecting Sparse Changes in High-Dimensional Heteroscedastic Data
• Diagonal Likelihood Ratio Test for Equality of Mean Vectors in High-Dimensional Data
• Bootstrapping ${\ell }_p$-Statistics in High Dimensions
• On Tests for Complete Independence of Normal Random Vectors
• Spatio-Temporal Big Data Analysis for Smart Grids Based on Random Matrix Theory: A Comprehensive Study
• High Dimensional Spatial Rank Test for Two-Sample Location Problem
• Scalar-Invariant Test for High-Dimensional Regression Coefficients
• Estimation and inference for high-dimensional non-sparse models
• Likelihood-based tests on linear hypotheses of large dimensional mean vectors with unequal covariance matrices
• Max-Type and Sum-Type Procedures for Online Change-Point Detection in the Mean of High-Dimensional Data
• Hybrid Regularisation of Functional Linear Models
• Significance Analysis of High-Dimensional, Low-Sample Size Partially Labeled Data
• Generalized Multivariate Signs for Nonparametric Hypothesis Testing in High Dimensions
• Conditional Mean and Quantile Dependence Testing in High Dimension
• Testing independence with high-dimensional correlated samples
• Two edge-count tests and relevance analysis in k high-dimensional samples
• On testing the equality of high dimensional mean vectors with unequal covariance matrices
• Poisson and Gaussian approximations of the power divergence family of statistics
• High-dimensional tests for spherical location and spiked covariance
• Optimal Sign Test for High Dimensional Location Parameters
• Mean vector testing for high dimensional dependent observations
• The general linear hypothesis testing problem for multivariate functional data with applications
• A Global Homogeneity Test for High-Dimensional Linear Regression
• High Dimensional Rank Tests for Sphericity
• Statistical inference with F-statistics when fitting simple models to high-dimensional data
• Multinomial Goodness-of-Fit Based on U-Statistics: High-Dimensional Asymptotic and Minimax Optimality
• A Note on High Dimensional Two Sample Mean Test
• High-dimensional inference on covariance structures via the extended cross-data-matrix methodology