Random Text, Zipf's Law, Critical Length,and Implications for Large Language Models

Vladimir Berman

We study a deliberately simple, fully non-linguistic model of text: a sequence of independent draws from a finite alphabet of letters plus a single space symbol. A word is defined…