The brevity law as a scaling law, and a possible origin of Zipf's law for word frequencies

Abstract

An important body of quantitative linguistics is constituted by a series of statistical laws about language usage. Despite the importance of these linguistic laws, some of them are poorly formulated, and, more importantly, there is no unified framework that encompasses all them. This paper presents a new perspective to establish a connection between different statistical linguistic laws. Characterizing each word type by two random variables, length (in number of characters) and absolute frequency, we show that the corresponding bivariate joint probability distribution shows a rich and precise phenomenology, with the type-length and the type-frequency distributions as its two marginals, and the conditional distribution of frequency at fixed length providing a clear formulation for the brevity-frequency phenomenon. The type-length distribution turns out to be well fitted by a gamma distribution (much better than with the previously proposed lognormal), and the conditional frequency distributions at fixed length display power-law-decay behavior with a fixed exponent $\alpha \simeq 1.4$ and a characteristic-frequency crossover that scales as an inverse power $\delta \simeq 2.8$ of length, which implies the fulfilment of a scaling law analogous to those found in the thermodynamics of critical phenomena. As a by-product, we find a possible model-free explanation for the origin of Zipf's law, which should arise as a mixture of conditional frequency distributions governed by the crossover length-dependent frequency.

arXiv commentSubmitted to special issue on Information Theory and Language

References in corpus (17)

• Power-law distributions in empirical data
• Scale-free Networks Well Done
• Large-scale analysis of Zipf's law in English texts
• Power-law size distributions in geoscience revisited
• Universality of rain event size distributions
• Compression and the origins of Zipf's law for word frequencies
• Statistical laws in linguistics
• Zipf's law for word frequencies: word forms versus lemmas in long texts
• Toward a statistical mechanics of four letter words
• A scaling law beyond Zipf's law and its relation to Heaps' law
• Scaling in the Timing of Extreme Events
• The distinct flavors of Zipf's law in the rank-size and in the size-distribution representations, and its maximum-likelihood fitting
• Log-log Convexity of Type-Token Growth in Zipf's Systems
• Text mixing shapes the anatomy of rank-frequency distributions: A modern Zipfian mechanics for natural language
• Non-characteristic Half-lives in Radioactive Decay
• Increasing power-law range in avalanche amplitude and energy distributions
• From Boltzmann to Zipf through Shannon and Jaynes

Cited by in corpus (11)

• Linguistic laws in biology
• Complex systems approach to natural language
• The distinct flavors of Zipf's law in the rank-size and in the size-distribution representations, and its maximum-likelihood fitting
• Lognormals, Power Laws and Double Power Laws in the Distribution of Frequencies of Harmonic Codewords from Classical Music
• Heaps' Law and Vocabulary Richness in the History of Classical Music Harmony
• Entropy and type-token ratio in gigaword corpora
• From Boltzmann to Zipf through Shannon and Jaynes
• On the class of coding optimality of human languages and the origins of Zipf's law
• Hearings and mishearings: decrypting the spoken word
• Zipf's laws of meaning in Catalan
• The Dynamics of Human and AI-Generated Language: How Semantics Fluctuates across Different Timescales