Exchangeable Testing Against an Unknown Benchmark

Abstract

We generate infinite binary exchangeable sequences by sequential comparison of data points against a latent benchmark. Assuming a prior distribution of the benchmark rank \(R_0\) within an unobserved group of players, we set up the Bayesian machinery which determines the posterior distribution of the running rank \(R_n\) in purely combinatorial terms, by counting insertion slots in a uniformly random ranking. This yields an explicit predictive probability of winning against the benchmark, which is a rational function of the counts. We identify some cases where the predictive probability has particularly simple form, notably a tournament related to the Topp--Leone distribution. The running benchmark rank is a Pólya urn with random initial composition, and its normalised value converges to a latent strength variable with polynomial density. In a projective time scale, the fluctuations of the normalised rank about this limit are Brownian, with the Wright--Fisher coefficient frozen at the limit, while the associated recurrent up-down Markov chain on a high level $n$ of the Pascal graph is approximated by a Wright--Fisher diffusion with a nonlinear drift expressed through the prior density.

arXiv comment27 pages