Power of Ordered Hypothesis Testing
arXiv:1606.01969
Abstract
Ordered testing procedures are multiple testing procedures that exploit a pre-specified ordering of the null hypotheses, from most to least promising. We analyze and compare the power of several recent proposals using the asymptotic framework of Li & Barber (2015). While accumulation tests including ForwardStop can be quite powerful when the ordering is very informative, they are asymptotically powerless when the ordering is weaker. By contrast, Selective SeqStep, proposed by Barber & Candès (2015), is much less sensitive to the quality of the ordering. We compare the power of these procedures in different régimes, concluding that Selective SeqStep dominates accumulation tests if either the ordering is weak or non-null hypotheses are sparse or weak. Motivated by our asymptotic analysis, we derive an improved version of Selective SeqStep which we call Adaptive SeqStep, analogous to Storey's improvement on the Benjamini-Hochberg procedure. We compare these methods using the GEOQuery data set analyzed by Li & Barber (2015) and find Adaptive SeqStep has favorable performance for both good and bad prior orderings.
18 pages. To appear at ICML 2016
References in corpus (4)
Cited by in corpus (10)
- STAR: A general interactive framework for FDR control under structural constraints
- SAFFRON: an adaptive algorithm for online control of the false discovery rate
- Multiple testing with the structure adaptive Benjamini-Hochberg algorithm
- Asynchronous Online Testing of Multiple Hypotheses
- AdaPT: An interactive procedure for multiple testing with side information
- NeuralFDR: Learning Discovery Thresholds from Hypothesis Features
- Multiple competition-based FDR control for peptide detection
- The Control of the False Discovery Rate in Fixed Sequence Multiple Testing
- Sequential Multiple Testing
- Using prior information to boost power in correlation structure support recovery