Covariate powered cross-weighted multiple testing

Abstract

A fundamental task in the analysis of datasets with many variables is screening for associations. This can be cast as a multiple testing task, where the objective is achieving high detection power while controlling type I error. We consider $m$ hypothesis tests represented by pairs $((P_i,X_i){)}_{1\le i\le m}$ of p-values $P_i$ and covariates $X_i$, such that $P_i\perp X_i$ if $H_i$ is null. Here, we show how to use information potentially available in the covariates about heterogeneities among hypotheses to increase power compared to conventional procedures that only use the $P_i$. To this end, we upgrade existing weighted multiple testing procedures through the Independent Hypothesis Weighting (IHW) framework to use data-driven weights that are calculated as a function of the covariates. Finite sample guarantees, e.g., false discovery rate (FDR) control, are derived from cross-weighting, a data-splitting approach that enables learning the weight-covariate function without overfitting as long as the hypotheses can be partitioned into independent folds, with arbitrary within-fold dependence. IHW has increased power compared to methods that do not use covariate information. A key implication of IHW is that hypothesis rejection in common multiple testing setups should not proceed according to the ranking of the p-values, but by an alternative ranking implied by the covariate-weighted p-values.

arXiv commentJournal of the Royal Statistical Society: Series B (Statistical Methodology) 2021

References in corpus (19)

• Synthesis of clathrate cerium superhydride CeH9 below 100 GPa with atomic hydrogen sublattice
• Controlling the false discovery rate via knockoffs
• A stochastic process approach to false discovery control
• Gene Hunting with Knockoffs for Hidden Markov Models
• Genome-Wide Significance Levels and Weighted Hypothesis Testing
• Simultaneous inference: When should hypothesis testing problems be combined?
• Two simple sufficient conditions for FDR control
• Optimal weighting for false discovery rate control
• Unsupervised empirical Bayesian multiple testing with external covariates
• Power-enhanced multiple decision functions controlling family-wise error and false discovery rates
• Comment: Microarrays, Empirical Bayes and the Two-Groups Model
• Beyond the E-value: stratified statistics for protein domain prediction
• Single-index modulated multiple testing
• Optimal Multiple Testing Under a Gaussian Prior on the Effect Sizes
• Covariate-Powered Empirical Bayes Estimation
• NeuralFDR: Learning Discovery Thresholds from Hypothesis Features
• Incorporation of Sparsity Information in Large-scale Multiple Two-sample $t$ Tests
• Knockoffs with Side Information
• Learning false discovery rates by fitting sigmoidal threshold functions

Cited by in corpus (7)

• Knockoffs with Side Information
• AdaPT-GMM: Powerful and robust covariate-assisted multiple testing
• Sharp multiple testing boundary for sparse sequences
• Large-Scale Multiple Testing of Composite Null Hypotheses Under Heteroskedasticity
• Analysis of error control in large scale two-stage multiple hypothesis testing
• Using prior information to boost power in correlation structure support recovery
• Treatment Effect Detection with Controlled FDR under Dependence for Large-Scale Experiments