On the exact Berk-Jones statistics and their p-value calculation

Abstract

Continuous goodness-of-fit testing is a classical problem in statistics. Despite having low power for detecting deviations at the tail of a distribution, the most popular test is based on the Kolmogorov-Smirnov statistic. While similar variance-weighted statistics, such as Anderson-Darling and the Higher Criticism statistic give more weight to tail deviations, as shown in various works, they still mishandle the extreme tails. As a viable alternative, in this paper we study some of the statistical properties of the exact $M_n$ statistics of Berk and Jones. We derive the asymptotic null distributions of $M_n,M_n^{+},M_n^{-}$, and further prove their consistency as well as asymptotic optimality for a wide range of rare-weak mixture models. Additionally, we present a new computationally efficient method to calculate $p$-values for any supremum-based one-sided statistic, including the one-sided $M_n^{+},M_n^{-}$ and $R_n^{+},R_n^{-}$ statistics of Berk and Jones and the Higher Criticism statistic. We illustrate our theoretical analysis with several finite-sample simulations.

29 pages, 3 figures, pdflatex; Minor revision

References in corpus (4)

• Goodness-of-fit tests via phi-divergences
• Higher criticism: $p$-values and criticism
• Goodness of fit tests in terms of local levels with special emphasis on higher criticism tests
• Fast calculation of boundary crossing probabilities for Poisson processes

Cited by in corpus (14)

• Comparing distributions by multiple testing across quantiles or CDF values
• On the cross-validation bias due to unsupervised pre-processing
• A study of periodograms standardized using training data sets and application to exoplanet detection
• 3D magneto-hydrodynamical simulations of stellar convective noise for improved exoplanet detection. I. Case of regularly sampled radial velocity observations
• Extended-support beta regression for $[0,1]$ responses
• Goodness of fit tests in terms of local levels with special emphasis on higher criticism tests
• Fast calculation of boundary crossing probabilities for Poisson processes
• Fast calculation of p-values for one-sided Kolmogorov-Smirnov type statistics
• Unification of Rare and Weak Detection Models using Moderate Deviations Analysis and Log-Chisquared P-values
• Distributions and Statistical Power of Optimal Signal-Detection Methods In Finite Cases
• Sparse Anomaly Detection Across Referentials: A Rank-Based Higher Criticism Approach
• Optimality of the max test for detecting sparse signals with Gaussian or heavier tail
• Higher criticism for rare and weak non-proportional hazard deviations in survival analysis
• Some detection tests for low complexity data models and unknown background distribution