Detection of correlations

Abstract

We consider the hypothesis testing problem of deciding whether an observed high-dimensional vector has independent normal components or, alternatively, if it has a small subset of correlated components. The correlated components may have a certain combinatorial structure known to the statistician. We establish upper and lower bounds for the worst-case (minimax) risk in terms of the size of the correlated subset, the level of correlation, and the structure of the class of possibly correlated sets. We show that some simple tests have near-optimal performance in many cases, while the generalized likelihood ratio test is suboptimal in some important cases.

Published in at http://dx.doi.org/10.1214/11-AOS964 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (7)

• Covariance regularization by thresholding
• Optimal rates of convergence for covariance matrix estimation
• Detection of an anomalous cluster in a network
• Innovated higher criticism for detecting sparse signals in correlated noise
• Searching for a trail of evidence in a maze
• On combinatorial testing problems
• Detection of Gauss-Markov Random Fields with Nearest-Neighbor Dependency

Cited by in corpus (6)

• Optimal detection of sparse principal components in high dimension
• Optimality and Sub-optimality of PCA I: Spiked Random Matrix Models
• Locally Most Powerful Invariant Tests for Correlation and Sphericity of Gaussian Vectors
• Tests alternative to higher criticism for high-dimensional means under sparsity and column-wise dependence
• Detecting positive correlations in a multivariate sample
• High-Temperature Structure Detection in Ferromagnets