Test Martingales, Bayes Factors and $p$-Values

Abstract

A nonnegative martingale with initial value equal to one measures evidence against a probabilistic hypothesis. The inverse of its value at some stopping time can be interpreted as a Bayes factor. If we exaggerate the evidence by considering the largest value attained so far by such a martingale, the exaggeration will be limited, and there are systematic ways to eliminate it. The inverse of the exaggerated value at some stopping time can be interpreted as a $p$-value. We give a simple characterization of all increasing functions that eliminate the exaggeration.

Published in at http://dx.doi.org/10.1214/10-STS347 the Statistical Science (http://www.imstat.org/sts/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (1)

• Insuring against loss of evidence in game-theoretic probability

Cited by in corpus (23)

• E-values: Calibration, combination, and applications
• Universal Inference
• Time-uniform, nonparametric, nonasymptotic confidence sequences
• Efficient quantification of experimental evidence against local realism
• Why optional stopping can be a problem for Bayesians
• (Nearly) optimal P-values for all Bell inequalities
• Certifying Quantum Randomness by Probability Estimation
• Optional Stopping with Bayes Factors: a categorization and extension of folklore results, with an application to invariant situations
• Quantum Randomness Generation by Probability Estimation with Classical Side Information
• Hypothesis testing with e-values
• Bell Inequalities for Continuously Emitting Sources
• Insuring against loss of evidence in game-theoretic probability
• Comparing Sequential Forecasters
• Post-selection inference for e-value based confidence intervals
• Anytime-Valid Linear Models and Regression Adjusted Causal Inference in Randomized Experiments
• ALL-IN meta-analysis: breathing life into living systematic reviews and prospective meta-analyses
• Anytime valid and asymptotically optimal inference driven by predictive recursion
• Bayes Factor Hypothesis Testing in Meta-Analyses: Practical Advantages and Methodological Considerations
• Performance of Test Supermartingale Confidence Intervals for the Success Probability of Bernoulli Trials
• Comment on Glenn Shafer's "Testing by betting"
• Sequential Monte-Carlo testing by betting
• Hypothesis testing for community structure in temporal networks using e-values
• On Confidence Sequences for Bounded Random Processes via Universal Gambling Strategies