Sequential stratified inference for the mean

Abstract

We develop conservative tests for the mean of a bounded population under stratified sampling and apply them to risk-limiting post-election audits. The tests are "anytime valid" under sequential sampling, allowing optional stopping in each stratum. Our core method expresses a global hypothesis about the population mean as a union of intersection hypotheses describing within-stratum means. It tests each intersection hypothesis using a test supermartingale (TSM) combined across strata by multiplication. A $P$-value for each intersection hypothesis is the reciprocal of that TSM, and the largest $P$-value in the union is a $P$-value for the global hypothesis. This approach has two primary moving parts: the rule selecting which stratum to draw from next, and the form of the TSM within each stratum. These rules may vary over intersection hypotheses. We construct intersection tests with the smallest expected sample size and propagate them to approximate an efficient global test. In instances that arise in auditing and other applications, its expected sample size is substantially smaller than that of previous methods.

24 pages, 5 figures, submitted to Annals of Applied Statistics