Correct ordering in the Zipf-Poisson ensemble

Abstract

We consider a Zipf--Poisson ensemble in which $X_i\sim\poi(Ni^{-α})$ for $\alpha >1$ and $N>0$ and integers $i\ge 1$. As $N\to \infty$ the first $n^{\prime }(N)$ random variables have their proper order $X_1>X_2>...>X_{n^{\prime }}$ relative to each other, with probability tending to 1 for $n^{\prime }$ up to $(AN/\log (N){)}^{1/(\alpha +2)}$ for an explicit constant $A(\alpha )\ge 3/4$. The rate $N^{1/(\alpha +2)}$ cannot be achieved. The ordering of the first $n^{\prime }(N)$ entities does not preclude $X_m>X_{n^{\prime }}$ for some interloping $m>n^{\prime }$. The first $n"$ random variables are correctly ordered exclusive of any interlopers, with probability tending to 1 if $n"\le (BN/\log (N){)}^{1/(\alpha +2)}$ for $B<A$. For a Zipf--Poisson model of the British National Corpus, which has a total word count of $100,000,000$, our result estimates that the 72 words with the highest counts are properly ordered.