On the performance of FDR control: Constraints and a partial solution

Abstract

The False Discovery Rate (FDR) paradigm aims to attain certain control on Type I errors with relatively high power for multiple hypothesis testing. The Benjamini--Hochberg (BH) procedure is a well-known FDR controlling procedure. Under a random effects model, we show that, in general, unlike the FDR, the positive FDR (pFDR) of the BH procedure cannot be controlled at an arbitrarily low level due to the limited evidence provided by the observations to separate false and true nulls. This results in a criticality phenomenon, which is characterized by a transition of the procedure's power from being positive to asymptotically 0 without any reduction in the pFDR, once the target FDR control level is below a positive critical value. To address the constraints on the power and pFDR control imposed by the criticality phenomenon, we propose a procedure which applies BH-type procedures at multiple locations in the domain of $p$-values. Both analysis and simulations show that the proposed procedure can attain substantially improved power and pFDR control.

Published in at http://dx.doi.org/10.1214/009053607000000037 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (5)

• Higher criticism for detecting sparse heterogeneous mixtures
• A stochastic process approach to false discovery control
• Estimating the proportion of false null hypotheses among a large number of independently tested hypotheses
• False discovery and false nondiscovery rates in single-step multiple testing procedures
• Sample size and positive false discovery rate control for multiple testing

Cited by in corpus (13)

• On the false discovery rate and an asymptotically optimal rejection curve
• Asymptotic Bayes-optimality under sparsity of some multiple testing procedures
• Simultaneous Control of All False Discovery Proportions in Large-Scale Multiple Hypothesis Testing
• Exact calculations for false discovery proportion with application to least favorable configurations
• On false discovery rate thresholding for classification under sparsity
• New procedures controlling the false discovery proportion via Romano-Wolf's heuristic
• Asymptotic properties of false discovery rate controlling procedures under independence
• False discovery rate control with multivariate $p$-values
• Simultaneous critical values for $t$-tests in very high dimensions
• A central limit theorem for the Benjamini-Hochberg false discovery proportion under a factor model
• Sample size and positive false discovery rate control for multiple testing
• A Scan Procedure for Multiple Testing
• Bayesian Multiple Testing Under Sparsity for Polynomial-Tailed Distributions