Testing statistical laws in complex systems

Abstract

The availability of large datasets requires an improved view on statistical laws in complex systems, such as Zipf's law of word frequencies, the Gutenberg-Richter law of earthquake magnitudes, or scale-free degree distribution in networks. In this paper we discuss how the statistical analysis of these laws are affected by correlations present in the observations, the typical scenario for data from complex systems. We first show how standard maximum-likelihood recipes lead to false rejections of statistical laws in the presence of correlations. We then propose a conservative method (based on shuffling and under-sampling the data) to test statistical laws and find that accounting for correlations leads to smaller rejection rates and larger confidence intervals on estimated parameters.

5-pages paper + 7-pages supplementary material

References in corpus (22)

• Emergence of scaling in random networks
• Power-law distributions in empirical data
• Power laws, Pareto distributions and Zipf's law
• The origin of bursts and heavy tails in human dynamics
• Scale-free networks are rare
• Problems with Fitting to the Power-Law Distribution
• Long-Term Clustering, Scaling, and Universality in the Temporal Occurrence of Earthquakes
• Statistical properties of sampled networks
• Fluctuation scaling in complex systems: Taylor's law and beyond
• Beyond word frequency: Bursts, lulls, and scaling in the temporal distributions of words
• Parameter estimation for power-law distributions by maximum likelihood methods
• Stochastic model for the vocabulary growth in natural languages
• Large-scale analysis of Zipf's law in English texts
• Is this scaling nonlinear?
• Sampling properties of random graphs: the degree distribution
• Statistical laws in linguistics
• Fitting Power-laws in empirical data with estimators that work for all exponents
• Rigorous statistical detection and characterization of a deviation from the Gutenberg-Richter distribution above magnitude 8 in subduction zones
• Probability Models for Degree Distributions of Protein Interaction Networks
• Goodness-of-Fit tests with Dependent Observations
• Universal Complex Structures in Written Language
• How likely is an i.i.d. degree sequence to be graphical?

Cited by in corpus (19)

• Robustness and resilience of complex networks
• Scale-free Networks Well Done
• True scale-free networks hidden by finite size effects
• Self-organization toward criticality by synaptic plasticity
• Linguistic laws in biology
• Good and bad predictions: Assessing and improving the replication of chaotic attractors by means of reservoir computing
• The distinct flavors of Zipf's law in the rank-size and in the size-distribution representations, and its maximum-likelihood fitting
• Disentangling the critical signatures of neural activity
• Spatial interactions in urban scaling laws
• A Master Equation for Power Laws
• Some recent advances in urban system science: models and data
• The emergence of scale-free fires in Australia
• On the emergence of Zipf's law in music
• Identifying time dependence in network growth
• Wave function network description and Kolmogorov complexity of quantum many-body systems
• Lognormals, Power Laws and Double Power Laws in the Distribution of Frequencies of Harmonic Codewords from Classical Music
• Impact of inter-city interactions on disease scaling
• Non-parametric power-law surrogates
• Tunable robustness in power-law inference