Global testing under sparse alternatives: ANOVA, multiple comparisons and the higher criticism

Abstract

Testing for the significance of a subset of regression coefficients in a linear model, a staple of statistical analysis, goes back at least to the work of Fisher who introduced the analysis of variance (ANOVA). We study this problem under the assumption that the coefficient vector is sparse, a common situation in modern high-dimensional settings. Suppose we have $p$ covariates and that under the alternative, the response only depends upon the order of $p^{1-\alpha }$ of those, $0\le \alpha \le 1$. Under moderate sparsity levels, that is, $0\le \alpha \le 1/2$, we show that ANOVA is essentially optimal under some conditions on the design. This is no longer the case under strong sparsity constraints, that is, $\alpha >1/2$. In such settings, a multiple comparison procedure is often preferred and we establish its optimality when $\alpha \ge 3/4$. However, these two very popular methods are suboptimal, and sometimes powerless, under moderately strong sparsity where $1/2<\alpha <3/4$. We suggest a method based on the higher criticism that is powerful in the whole range $\alpha >1/2$. This optimality property is true for a variety of designs, including the classical (balanced) multi-way designs and more modern "$p>n$" designs arising in genetics and signal processing. In addition to the standard fixed effects model, we establish similar results for a random effects model where the nonzero coefficients of the regression vector are normally distributed.

Published in at http://dx.doi.org/10.1214/11-AOS910 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (1)

• Properties of higher criticism under strong dependence

Cited by in corpus (55)

• Optimal detection of sparse principal components in high dimension
• Higher Criticism for Large-Scale Inference, Especially for Rare and Weak Effects
• Detection of a sparse submatrix of a high-dimensional noisy matrix
• Confidence sets in sparse regression
• Optimality and Sub-optimality of PCA I: Spiked Random Matrix Models
• MORESANE: MOdel REconstruction by Synthesis-ANalysis Estimators. A sparse deconvolution algorithm for radio interferometric imaging
• Minimax estimation of linear and quadratic functionals on sparsity classes
• Computational Lower Bounds for Sparse PCA
• Optimality and Sub-optimality of PCA for Spiked Random Matrices and Synchronization
• Hypothesis testing for high-dimensional sparse binary regression
• Cauchy combination test: a powerful test with analytic p-value calculation under arbitrary dependency structures
• BET on Independence
• Detection boundary and Higher Criticism approach for rare and weak genetic effects
• Optimal classification in sparse Gaussian graphic model
• Regularized Estimation and Testing for High-Dimensional Multi-Block Vector-Autoregressive Models
• Higher Criticism to Compare Two Large Frequency Tables, with sensitivity to Possible Rare and Weak Differences
• Robust control of varying weak hyperspectral target detection with sparse non-negative representation
• Distribution-Free Tests for Sparse Heterogeneous Mixtures
• Sparse High-Dimensional Linear Regression. Algorithmic Barriers and a Local Search Algorithm
• Ensemble methods for testing a global null
• Fast calculation of p-values for one-sided Kolmogorov-Smirnov type statistics
• Higher Criticism for Discriminating Word-Frequency Tables and Testing Authorship
• Comparing a Large Number of Multivariate Distributions
• Optimal Detection For Sparse Mixtures
• Multiple Support Recovery Using Very Few Measurements Per Sample
• Testing High Dimensional Mean Under Sparsity
• Lattice partition recovery with dyadic CART
• Multiple testing when many $p$-values are uniformly conservative, with application to testing qualitative interaction in educational interventions
• Randomized tests for high-dimensional regression: A more efficient and powerful solution
• Sharp multiple testing boundary for sparse sequences
• Heritability estimation in high dimensional linear mixed models
• Anomaly Detection for a Large Number of Streams: A Permutation-Based Higher Criticism Approach
• Estimation of the $l_2$-norm and testing in sparse linear regression with unknown variance
• Unification of Rare and Weak Detection Models using Moderate Deviations Analysis and Log-Chisquared P-values
• Moderate-Dimensional Inferences on Quadratic Functionals in Ordinary Least Squares
• Residual permutation test for regression coefficient testing
• Global testing against sparse alternatives in time-frequency analysis
• Blockwise and coordinatewise thresholding to combine tests of different natures in modern ANOVA
• On the Minimum Attainable Risk in Permutation Invariant Problems
• Generalized Goodness-Of-Fit Tests for Correlated Data
• Optimality of the max test for detecting sparse signals with Gaussian or heavier tail
• Sparse Anomaly Detection Across Referentials: A Rank-Based Higher Criticism Approach
• Optimal partition recovery in general graphs
• Two-sample testing of high-dimensional linear regression coefficients via complementary sketching
• Honest confidence sets for high-dimensional regression by projection and shrinkage
• Optimal Estimation of A Quadratic Functional and Detection of Simultaneous Signals
• Distributions and Statistical Power of Optimal Signal-Detection Methods In Finite Cases
• Variable Selection via Adaptive False Negative Control in Linear Regression
• Global Testing Against Sparse Alternatives under Ising Models
• Partial Correlation Screening for Estimating Large Precision Matrices, with Applications to Classification
• Optimal detection of weak positive latent dependence between two sequences of multiple tests
• Global testing under the sparse alternatives for single index models
• Higher criticism for rare and weak non-proportional hazard deviations in survival analysis
• Superconsistency of Tests in High Dimensions
• A Global Homogeneity Test for High-Dimensional Linear Regression