Interactive Martingale Tests for the Global Null

Abstract

Global null testing is a classical problem going back about a century to Fisher's and Stouffer's combination tests. In this work, we present simple martingale analogs of these classical tests, which are applicable in two distinct settings: (a) the online setting in which there is a possibly infinite sequence of $p$-values, and (b) the batch setting, where one uses prior knowledge to preorder the hypotheses. Through theory and simulations, we demonstrate that our martingale variants have higher power than their classical counterparts even when the preordering is only weakly informative. Finally, using a recent idea of "masking" $p$-values, we develop a novel interactive test for the global null that can take advantage of covariates and repeated user guidance to create a data-adaptive ordering that achieves higher detection power against structured alternatives.

arXiv comment48 pages, 16 figures

References in corpus (6)

• Higher Criticism for Large-Scale Inference, Especially for Rare and Weak Effects
• An adaptively weighted statistic for detecting differential gene expression when combining multiple transcriptomic studies
• Time-uniform Chernoff bounds via nonnegative supermartingales
• STAR: A general interactive framework for FDR control under structural constraints
• Admissible anytime-valid sequential inference must rely on nonnegative martingales
• Properties of adaptively weighted Fisher's method

Cited by in corpus (3)

• E-values: Calibration, combination, and applications
• Information Flow in Computational Systems
• Familywise Error Rate Control by Interactive Unmasking