Zipf and Heaps laws from dependency structures in component systems

Abstract

Complex natural and technological systems can be considered, on a coarse-grained level, as assemblies of elementary components: for example, genomes as sets of genes, or texts as sets of words. On one hand, the joint occurrence of components emerges from architectural and specific constraints in such systems. On the other hand, general regularities may unify different systems, such as the broadly studied Zipf and Heaps laws, respectively concerning the distribution of component frequencies and their number as a function of system size. Dependency structures (i.e., directed networks encoding the dependency relations between the components in a system) were proposed recently as a possible organizing principles underlying some of the regularities observed. However, the consequences of this assumption were explored only in binary component systems, where solely the presence or absence of components is considered, and multiple copies of the same component are not allowed. Here, we consider a simple model that generates, from a given ensemble of dependency structures, a statistical ensemble of sets of components, allowing for components to appear with any multiplicity. Our model is a minimal extension that is memoryless, and therefore accessible to analytical calculations. A mean-field analytical approach (analogous to the "Zipfian ensemble" in the linguistics literature) captures the relevant laws describing the component statistics as we show by comparison with numerical computations. In particular, we recover a power-law Zipf rank plot, with a set of core components, and a Heaps law displaying three consecutive regimes (linear, sub-linear and saturating) that we characterize quantitatively.

References in corpus (22)

• Power laws, Pareto distributions and Zipf's law
• Are biological systems poised at criticality?
• Nonequilibrium Statistical Mechanics of the Zero-Range Process and Related Models
• Stretched Exponential Relaxation Arising from a Continuous Sum of Exponential Decays
• Information-based fitness and the emergence of criticality in living systems
• Network dynamics of innovation processes
• Are there laws of genome evolution?
• Stochastic model for the vocabulary growth in natural languages
• Zipf's Law Leads to Heaps' Law: Analyzing Their Relation in Finite-Size Systems
• Toolbox model of evolution of prokaryotic metabolic networks and their regulation
• When do generalized entropies apply? How phase space volume determines entropy
• Understanding scaling through history-dependent processes with collapsing sample space
• Statistical laws in linguistics
• Scaling laws and fluctuations in the statistics of word frequencies
• Universal distribution of component frequencies in biological and technological systems
• Evolution of a Modular Software Network
• Log-log Convexity of Type-Token Growth in Zipf's Systems
• Joint scaling laws in functional and evolutionary categories in prokaryotic genomes
• Spatial aggregation and the species-area relationship across scales
• Statistics of shared components in complex component systems
• Cross-scale ecological theory sheds light on the maintenance of biodiversity
• Mean-field methods in evolutionary duplication-innovation-loss models for the genome-level repertoire of protein domains

Cited by in corpus (10)

• Interacting discovery processes on complex networks
• A Master Equation for Power Laws
• Statistical learning theory of structured data
• Heaps' law, statistics of shared components and temporal patterns from a sample-space-reducing process
• Generalization from correlated sets of patterns in the perceptron
• Inversion dynamics of class manifolds in deep learning reveals tradeoffs underlying generalisation
• Killer Technologies: the destructive creation in the technical change
• A new concept of technology with systemic-purposeful perpsective: theory, examples and empirical application
• Cyclical phenomena in technological change
• Quadratic Term Correction on Heaps' Law