Zipf's law and criticality in multivariate data without fine-tuning

Abstract

The joint probability distribution of many degrees of freedom in biological systems, such as firing patterns in neural networks or antibody sequence composition in zebrafish, often follow Zipf's law, where a power law is observed on a rank-frequency plot. This behavior has recently been shown to imply that these systems reside near to a unique critical point where the extensive parts of the entropy and energy are exactly equal. Here we show analytically, and via numerical simulations, that Zipf-like probability distributions arise naturally if there is an unobserved variable (or variables) that affects the system, e. g. for neural networks an input stimulus that causes individual neurons in the network to fire at time-varying rates. In statistics and machine learning, these models are called latent-variable or mixture models. Our model shows that no fine-tuning is required, i.e. Zipf's law arises generically without tuning parameters to a point, and gives insight into the ubiquity of Zipf's law in a wide range of systems.

5 pages, 3 figures

References in corpus (5)

• Power-law distributions in empirical data
• The large deviation approach to statistical mechanics
• Emergent complex neural dynamics
• Identification of direct residue contacts in protein-protein interaction by message passing
• Statistical inference of the generation probability of T-cell receptors from sequence repertoires

Cited by in corpus (40)

• A high-bias, low-variance introduction to Machine Learning for physicists
• Colloquium: Criticality and dynamical scaling in living systems
• Inverse statistical problems: from the inverse Ising problem to data science
• Power laws and Self-Organized Criticality in Theory and Nature
• Power-law statistics and universal scaling in the absence of criticality
• Computational strategies for dissecting the high-dimensional complexity of adaptive immune repertoires
• Augmenting adaptive immunity: progress and challenges in the quantitative engineering and analysis of adaptive immune receptor repertoires
• Signatures of criticality arise in simple neural population models with correlations
• On the sufficiency of pairwise interactions in maximum entropy models of biological networks
• Statistical Criticality arises in Most Informative Representations
• Latent dynamical variables produce signatures of spatiotemporal criticality in large biological systems
• Rényi entropy, abundance distribution and the equivalence of ensembles
• Quantifying Relevance in Learning and Inference
• Maximal Diversity and Zipf's Law
• Zipf's and Taylor's Laws
• Phase Transitions and Criticality in the Collective Behavior of Animals -- Self-organization and biological function
• Minimum Description Length codes are critical
• Universality of biochemical feedback and its application to immune cells
• Fluctuating landscapes and heavy tails in animal behavior
• Ensemble Inhibition and Excitation in the Human Cortex: an Ising Model Analysis with Uncertainties
• The Structured `Low Temperature' Phase of the Retinal Population Code
• Breakdown of random matrix universality in Markov models
• Statistical properties of large data sets with linear latent features
• Nonlinear Stochastic Dynamics of Complex Systems, II: Potential of Entropic Force in Markov Systems with Nonequilibrium Steady State, Generalized Gibbs Function and Criticality
• Inferring couplings in networks across order-disorder phase transitions
• Itinerant conductance in fuse-antifuse networks
• Optimal encoding in stochastic latent-variable Models
• The statistical mechanics of Twitter
• Extrinsic vs Intrinsic Criticality in Systems with Many Components
• Scale-invariant representation of machine learning
• Semiparametric energy-based probabilistic models
• Criticality & Deep Learning I: Generally Weighted Nets
• Resolution and Relevance Trade-offs in Deep Learning
• Genetic noise mechanism for power-law switching in bacterial flagellar motors
• Statistical Patterns in the Equations of Physics and the Emergence of a Meta-Law of Nature
• Unsupervised Bayesian Ising Approximation for revealing the neural dictionary in songbirds
• Strong consensus on US Supreme Court spans a century
• Universal emergence of local Zipf-Mandelbrot law
• On the scaling of avalanche shape and activity power spectrum in neuronal networks
• Maximal Relevance and Optimal Learning Machines