Analysis of error control in large scale two-stage multiple hypothesis testing

Abstract

When dealing with the problem of simultaneously testing a large number of null hypotheses, a natural testing strategy is to first reduce the number of tested hypotheses by some selection (screening or filtering) process, and then to simultaneously test the selected hypotheses. The main advantage of this strategy is to greatly reduce the severe effect of high dimensions. However, the first screening or selection stage must be properly accounted for in order to maintain some type of error control. In this paper, we will introduce a selection rule based on a selection statistic that is independent of the test statistic when the tested hypothesis is true. Combining this selection rule and the conventional Bonferroni procedure, we can develop a powerful and valid two-stage procedure. The introduced procedure has several nice properties: (i) it completely removes the selection effect; (ii) it reduces the multiplicity effect; (iii) it does not "waste" data while carrying out both selection and testing. Asymptotic power analysis and simulation studies illustrate that this proposed method can provide higher power compared to usual multiple testing methods while controlling the Type 1 error rate. Optimal selection thresholds are also derived based on our asymptotic analysis.

arXiv comment50 pages, 5 figures

References in corpus (9)

• Higher criticism for detecting sparse heterogeneous mixtures
• Optimal Inference After Model Selection
• Higher Criticism for Large-Scale Inference, Especially for Rare and Weak Effects
• Covariate powered cross-weighted multiple testing
• Selective Sequential Model Selection
• Multiple testing with the structure adaptive Benjamini-Hochberg algorithm
• Selective inference with a randomized response
• AdaPT: An interactive procedure for multiple testing with side information
• Asymptotics of selective inference