Size, power and false discovery rates

Abstract

Modern scientific technology has provided a new class of large-scale simultaneous inference problems, with thousands of hypothesis tests to consider at the same time. Microarrays epitomize this type of technology, but similar situations arise in proteomics, spectroscopy, imaging, and social science surveys. This paper uses false discovery rate methods to carry out both size and power calculations on large-scale problems. A simple empirical Bayes approach allows the false discovery rate (fdr) analysis to proceed with a minimum of frequentist or Bayesian modeling assumptions. Closed-form accuracy formulas are derived for estimated false discovery rates, and used to compare different methodologies: local or tail-area fdr's, theoretical, permutation, or empirical null hypothesis estimates. Two microarray data sets as well as simulations are used to evaluate the methodology, the power diagnostics showing why nonnull cases might easily fail to appear on a list of ``significant'' discoveries.

Published in at http://dx.doi.org/10.1214/009053606000001460 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (2)

• Needles and straw in haystacks: Empirical Bayes estimates of possibly sparse sequences
• A stochastic process approach to false discovery control

Cited by in corpus (47)

• Microarrays, Empirical Bayes and the Two-Groups Model
• On the false discovery rate and an asymptotically optimal rejection curve
• Simultaneous inference: When should hypothesis testing problems be combined?
• CAGI, the Critical Assessment of Genome Interpretation, establishes progress and prospects for computational genetic variant interpretation methods
• Are a set of microarrays independent of each other?
• Multiple hypothesis testing adjusted for latent variables, with an application to the AGEMAP gene expression data
• Power-enhanced multiple decision functions controlling family-wise error and false discovery rates
• Task-related edge density (TED) - a new method for revealing large-scale network formation in fMRI data of the human brain
• Detection boundary and Higher Criticism approach for rare and weak genetic effects
• Single-index modulated multiple testing
• Empirical null and false discovery rate inference for exponential families
• Resolving conflicts between statistical methods by probability combination: Application to empirical Bayes analyses of genomic data
• Why we (usually) don't have to worry about multiple comparisons
• Estimating the null distribution for conditional inference and genome-scale screening
• Large-scale interval and point estimates from an empirical Bayes extension of confidence posteriors
• AdaPT: An interactive procedure for multiple testing with side information
• Solving the Empirical Bayes Normal Means Problem with Correlated Noise
• Multiple Testing in Nonparametric Hidden Markov Models: An Empirical Bayes Approach
• Adaptive Sensing for Estimation of Structured Sparse Signals
• A Generalized Benjamini-Hochberg Procedure for Multivariate Hypothesis Testing
• Sample size and positive false discovery rate control for multiple testing
• Strong approximations of level exceedences related to multiple hypothesis testing
• Improved Inference of Gaussian Mixture Copula Model for Clustering and Reproducibility Analysis using Automatic Differentiation
• Inference with Transposable Data: Modeling the Effects of Row and Column Correlations
• Rejoinder: Microarrays, Empirical Bayes and the Two-Groups Model
• Estimating the number and effect sizes of non-null hypotheses
• RANK: Large-Scale Inference with Graphical Nonlinear Knockoffs
• HierLPR: Decision making in hierarchical multi-label classification with local precision rates
• Nonparametric Empirical Bayes Estimation and Testing for Sparse and Heteroscedastic Signals
• Empirical Bayes cumulative $\ell$-value multiple testing procedure for sparse sequences
• A semiparametric mixture method for local false discovery rate estimation
• Estimation of a semiparametric contaminated regression model
• CDfdr: A Comparison Density Approach to Local False Discovery Rate Estimation
• Identification of Signal, Noise, and Indistinguishable Subsets in High-Dimensional Data Analysis
• Bayesian Hierarchical Models for High-Dimensional Mediation Analysis with Coordinated Selection of Correlated Mediators
• Epigenome-wide association study and integrative analysis with the transcriptome based on GWAS summary statistics
• Network induced large correlation matrix estimation
• False discovery rate control under reduced precision computation for analysis of neuroimaging data
• Accounting for total variation and robustness in profiling health care providers
• Heterocedasticity-Adjusted Ranking and Thresholding for Large-Scale Multiple Testing
• A Flexible Procedure for Mixture Proportion Estimation in Positive-Unlabeled Learning
• Estimation of a Two-component Mixture Model with Applications to Multiple Testing
• Data volume and power of multiple tests with small sample size per null
• Exploratory data analysis for large-scale multiple testing problems and its application in gene expression studies
• Extending the Patra-Sen Approach to Estimating the Background Component in a Two-Component Mixture Model
• Maximum Likelihood Estimation of a Semiparametric Two-component Mixture Model using Log-concave Approximation
• Where to find needles in a haystack?