Goodness of fit tests in terms of local levels with special emphasis on higher criticism tests

Abstract

Instead of defining goodness of fit (GOF) tests in terms of their test statistics, we present an alternative method by introducing the concept of local levels, which indicate high or low local sensitivity of a test. Local levels can act as a starting point for the construction of new GOF tests. We study the behavior of local levels when applied to some well-known GOF tests such as Kolmogorov-Smirnov (KS) tests, higher criticism (HC) tests and tests based on phi-divergences. The main focus is on a rigorous characterization of the asymptotic behavior of local levels of the original HC tests which leads to several further asymptotic results for local levels of other GOF tests including GOF tests with equal local levels. While local levels of KS tests, which are related to the central range, are asymptotically strictly larger than zero, all local levels of HC tests converge to zero as the sample size increases. Consequently, there exists no asymptotic level $\alpha$ GOF test such that all local levels are asymptotically bounded away from zero. Finally, by means of numerical computations we compare classical KS and HC tests to a GOF test with equal local levels.

arXiv commentPublished at http://dx.doi.org/10.3150/14-BEJ694 in the Bernoulli (http://isi.cbs.nl/bernoulli/) by the International Statistical Institute/Bernoulli Society (http://isi.cbs.nl/BS/bshome.htm)

References in corpus (8)

• Higher criticism for detecting sparse heterogeneous mixtures
• Innovated higher criticism for detecting sparse signals in correlated noise
• Goodness-of-fit tests via phi-divergences
• On the false discovery rate and an asymptotically optimal rejection curve
• Properties of higher criticism under strong dependence
• Estimation and confidence sets for sparse normal mixtures
• On the exact Berk-Jones statistics and their p-value calculation
• Feature selection by Higher Criticism thresholding: optimal phase diagram

Cited by in corpus (4)

• On the exact Berk-Jones statistics and their p-value calculation
• Only Closed Testing Procedures are Admissible for Controlling False Discovery Proportions
• A study of periodograms standardized using training data sets and application to exoplanet detection
• Active set algorithms for estimating shape-constrained density ratios