FACT: Fast closed testing for exchangeable local tests
arXiv:1806.10163
Abstract
Multiple hypothesis testing problems arise naturally in science. In this paper, we introduce the new Fast Closed Testing (FACT) method for multiple testing, controlling the family-wise error rate. This error rate is state of the art in many important application areas, and is preferred to false discovery rate control for many reasons, including that it leads to stronger reproducibility. The closure principle rejects an individual hypothesis if all global nulls of subsets containing it are rejected using some test statistics. It takes exponential time in the worst case. When the tests are symmetric and monotone, our method is an exact algorithm for computing the closure, quadratic in the number of tests, and linear in the number of discoveries. Our framework generalizes most examples of closed testing such as Holm's and the Bonferroni method. As a special case of our method, we propose the Simes-higher criticism fusion test, which is powerful for detecting both a few strong signals, and also many moderate signals.
arXiv commentThis version fixes an indexing error in the calculation of adjusted p-values, pointed out in arXiv:1912.06116. To appear in Biometrika
References in corpus (5)
- Can parametric statistical methods be trusted for fMRI based group studies?
- Higher criticism for detecting sparse heterogeneous mixtures
- The sequential rejection principle of familywise error control
- Optimal Multiple Testing Under a Gaussian Prior on the Effect Sizes
- A shortcut for Hommel's procedure in linearithmic time