Higher criticism: $p$-values and criticism

Abstract

This paper compares the higher criticism statistic (Donoho and Jin [Ann. Statist. 32 (2004) 962-994]), a modification of the higher criticism statistic also suggested by Donoho and Jin, and two statistics of the Berk-Jones [Z. Wahrsch. Verw. Gebiete 47 (1979) 47-59] type. New approximations to the significance levels of the statistics are derived, and their accuracy is studied by simulations. By numerical examples it is shown that over a broad range of sample sizes the Berk-Jones statistics have a better power function than the higher criticism statistics to detect sparse mixtures. The applications suggested by Meinshausen and Rice [Ann. Statist. 34 (2006) 373-393], to find lower confidence bounds for the number of false hypotheses, and by Jeng, Cai and Li [Biometrika 100 (2013) 157-172], to detect copy number variants, are also studied.

Published at http://dx.doi.org/10.1214/15-AOS1312 in the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (5)

• Higher criticism for detecting sparse heterogeneous mixtures
• Estimating the proportion of false null hypotheses among a large number of independently tested hypotheses
• Sequential multi-sensor change-point detection
• Goodness-of-fit tests via phi-divergences
• Detecting simultaneous variant intervals in aligned sequences

Cited by in corpus (9)

• Higher Criticism for Large-Scale Inference, Especially for Rare and Weak Effects
• Simulation-Based Hypothesis Testing of High Dimensional Means Under Covariance Heterogeneity
• On the exact Berk-Jones statistics and their p-value calculation
• A study of periodograms standardized using training data sets and application to exoplanet detection
• Fast calculation of boundary crossing probabilities for Poisson processes
• Fast calculation of p-values for one-sided Kolmogorov-Smirnov type statistics
• Higher Criticism for Discriminating Word-Frequency Tables and Testing Authorship
• Global testing against sparse alternatives in time-frequency analysis
• Superconsistency of Tests in High Dimensions