The Future of Indirect Evidence

Abstract

Familiar statistical tests and estimates are obtained by the direct observation of cases of interest: a clinical trial of a new drug, for instance, will compare the drug's effects on a relevant set of patients and controls. Sometimes, though, indirect evidence may be temptingly available, perhaps the results of previous trials on closely related drugs. Very roughly speaking, the difference between direct and indirect statistical evidence marks the boundary between frequentist and Bayesian thinking. Twentieth-century statistical practice focused heavily on direct evidence, on the grounds of superior objectivity. Now, however, new scientific devices such as microarrays routinely produce enormous data sets involving thousands of related situations, where indirect evidence seems too important to ignore. Empirical Bayes methodology offers an attractive direct/indirect compromise. There is already some evidence of a shift toward a less rigid standard of statistical objectivity that allows better use of indirect evidence. This article is basically the text of a recent talk featuring some examples from current practice, with a little bit of futuristic speculation.

This paper commented in: [arXiv:1012.1414], [arXiv:1012.1423], [arXiv:1012.1479]. Rejoinder in [arXiv:1012.1489]. Published in at http://dx.doi.org/10.1214/09-STS308 the Statistical Science (http://www.imstat.org/sts/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (2)

• Microarrays, Empirical Bayes and the Two-Groups Model
• Simultaneous inference: When should hypothesis testing problems be combined?

Cited by in corpus (10)

• Estimation of COVID-19 spread curves integrating global data and borrowing information
• Statistical inference optimized with respect to the observed sample for single or multiple comparisons
• Measuring support for a hypothesis about a random parameter without estimating its unknown prior
• Large-Scale Multiple Hypothesis Testing with the Normal-Beta Prime Prior
• The Inverse Gamma-Gamma Prior for Optimal Posterior Contraction and Multiple Hypothesis Testing
• Smaller $p$-values via indirect information
• Inferring Multiple Graphical Structures
• Finding Important Genes from High-Dimensional Data: An Appraisal of Statistical Tests and Machine-Learning Approaches
• Bayes Multiple Decision Functions
• On the future of astrostatistics: statistical foundations and statistical practice