Simultaneous Control of All False Discovery Proportions in Large-Scale Multiple Hypothesis Testing
arXiv:1611.06739 · doi:10.1093/biomet/asz041
Abstract
Closed testing procedures are classically used for familywise error rate (FWER) control, but they can also be used to obtain simultaneous confidence bounds for the false discovery proportion (FDP) in all subsets of the hypotheses. In this paper we investigate the special case of closed testing with Simes local tests. We construct a novel fast and exact shortcut which we use to investigate the power of this method when the number of hypotheses goes to infinity. We show that, if a minimal amount of signal is present, the average power to detect false hypotheses at any desired FDP level does not vanish. Additionally, we show that the confidence bounds for FDP are consistent estimators for the true FDP for every non-vanishing subset. For the case of a finite number of hypotheses, we show connections between Simes-based closed testing and the procedure of Benjamini and Hochberg.
References in corpus (10)
- Higher criticism for detecting sparse heterogeneous mixtures
- Valid post-selection inference
- Generalizations of the Familywise Error Rate
- A stochastic process approach to false discovery control
- Multiple Testing for Exploratory Research
- On the Simes inequality and its generalization
- On the performance of FDR control: Constraints and a partial solution
- A shortcut for Hommel's procedure in linearithmic time
- Inferactive data analysis
- The FDR-Linking Theorem
Cited by in corpus (11)
- Permutation-based simultaneous confidence bounds for the false discovery proportion
- Simultaneous high-probability bounds on the false discovery proportion in structured, regression, and online settings
- Only Closed Testing Procedures are Admissible for Controlling False Discovery Proportions
- Confidence and discoveries with e-values
- Permutation-Based True Discovery Guarantee by Sum Tests
- FDP control in multivariate linear models using the bootstrap
- The Online Closure Principle
- Simultaneous directional inference
- Pathway Testing in Metabolomics with Globaltest, Allowing Post Hoc Choice of Pathways
- Localizing differences in smooths with simultaneous confidence bounds on the true discovery proportion
- The Lévy combination test