The distinct flavors of Zipf's law in the rank-size and in the size-distribution representations, and its maximum-likelihood fitting

Abstract

In the last years, researchers have realized the difficulties of fitting power-law distributions properly. These difficulties are higher in Zipf's systems, due to the discreteness of the variables and to the existence of two representations for these systems, i.e., two versions about which is the random variable to fit. The discreteness implies that a power law in one of the representations is not a power law in the other, and vice versa. We generate synthetic power laws in both representations and apply a state-of-the-art fitting method (based on maximum-likelihood plus a goodness-of-fit test) for each of the two random variables. It is important to stress that the method does not fit the whole distribution, but the tail, understood as the part of a distribution above a cut-off that separates non-power-law behavior from power-law behavior. We find that, no matter which random variable is power-law distributed, the rank-size representation is not adequate for fitting, whereas the representation in terms of the distribution of sizes leads to the recovery of the simulated exponents, may be with some bias.

References in corpus (10)

• Power-law distributions in empirical data
• A Poissonian explanation for heavy-tails in e-mail communication
• Languages cool as they expand: Allometric scaling and the decreasing need for new words
• Parameter estimation for power-law distributions by maximum likelihood methods
• Zipf's Law Leads to Heaps' Law: Analyzing Their Relation in Finite-Size Systems
• Universality of rain event size distributions
• Testing statistical laws in complex systems
• The brevity law as a scaling law, and a possible origin of Zipf's law for word frequencies
• Truncated lognormal distributions and scaling in the size of naturally defined population clusters
• A practical recipe to fit discrete power-law distributions

Cited by in corpus (7)

• Linguistic laws in biology
• The brevity law as a scaling law, and a possible origin of Zipf's law for word frequencies
• Maximal Diversity and Zipf's Law
• Lognormals, Power Laws and Double Power Laws in the Distribution of Frequencies of Harmonic Codewords from Classical Music
• Heaps' Law and Vocabulary Richness in the History of Classical Music Harmony
• From Boltzmann to Zipf through Shannon and Jaynes
• Finite-size scaling of human-population distributions over fixed-size cells and its relation to fractal spatial structure