E-values: Calibration, combination, and applications

Abstract

Multiple testing of a single hypothesis and testing multiple hypotheses are usually done in terms of p-values. In this paper we replace p-values with their natural competitor, e-values, which are closely related to betting, Bayes factors, and likelihood ratios. We demonstrate that e-values are often mathematically more tractable; in particular, in multiple testing of a single hypothesis, e-values can be merged simply by averaging them. This allows us to develop efficient procedures using e-values for testing multiple hypotheses.

48 pages, 5 figures, 4 algorithms. A new title and improved presentation

References in corpus (8)

• Multiple Testing for Exploratory Research
• Universal Inference
• Time-uniform, nonparametric, nonasymptotic confidence sequences
• Test Martingales, Bayes Factors and $p$-Values
• Testing hypotheses via a mixture estimation model
• The Language of Betting as a Strategy for Statistical and Scientific Communication
• Interactive Martingale Tests for the Global Null
• FACT: Fast closed testing for exchangeable local tests

Cited by in corpus (15)

• Testing randomness
• Evaluating probabilistic forecasts of extremes using continuous ranked probability score distributions
• Confidence and discoveries with e-values
• Permutation-Based True Discovery Guarantee by Sum Tests
• Conformal e-prediction
• Comparing Sequential Forecasters
• Rank-transformed subsampling: inference for multiple data splitting and exchangeable p-values
• Post-selection inference for e-value based confidence intervals
• False discovery rate control with e-values
• Simultaneous Selection of Multiple Important Single Nucleotide Polymorphisms in Familial Genome Wide Association Studies Data
• ALL-IN meta-analysis: breathing life into living systematic reviews and prospective meta-analyses
• Comment on Glenn Shafer's "Testing by betting"
• Testing exchangeability: fork-convexity, supermartingales, and e-processes
• Superconsistency of Tests in High Dimensions
• Inference for Large Panel Data with Many Covariates