Power-law distributions in binned empirical data

Abstract

Many man-made and natural phenomena, including the intensity of earthquakes, population of cities and size of international wars, are believed to follow power-law distributions. The accurate identification of power-law patterns has significant consequences for correctly understanding and modeling complex systems. However, statistical evidence for or against the power-law hypothesis is complicated by large fluctuations in the empirical distribution's tail, and these are worsened when information is lost from binning the data. We adapt the statistically principled framework for testing the power-law hypothesis, developed by Clauset, Shalizi and Newman, to the case of binned data. This approach includes maximum-likelihood fitting, a hypothesis test based on the Kolmogorov--Smirnov goodness-of-fit statistic and likelihood ratio tests for comparing against alternative explanations. We evaluate the effectiveness of these methods on synthetic binned data with known structure, quantify the loss of statistical power due to binning, and apply the methods to twelve real-world binned data sets with heavy-tailed patterns.

Published in at http://dx.doi.org/10.1214/13-AOAS710 the Annals of Applied Statistics (http://www.imstat.org/aoas/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (5)

• Power-law distributions in empirical data
• On the Frequency of Severe Terrorist Events
• Power-law distributions in binned empirical data
• Estimating the historical and future probabilities of large terrorist events
• The variability of tidewater-glacier calving: origin of event-size and interval distributions

Cited by in corpus (45)

• Power-law distributions in binned empirical data
• Scale-free Resilience of Real Traffic Jams
• Understanding star formation in molecular clouds II. Signatures of gravitational collapse of IRDCs
• Fitting Power-laws in empirical data with estimators that work for all exponents
• Modelling size distributions of marine plastics under the influence of continuous cascading fragmentation
• Inferring network properties from time series using transfer entropy and mutual information: validation of multivariate versus bivariate approaches
• Understanding star formation in molecular clouds IV. Column density PDFs from quiescent to massive molecular clouds
• Quantitative analysis of the evolution of novelty in cinema through crowdsourced keywords
• Theoretical foundations of studying criticality in the brain
• Traditional and Heavy-Tailed Self Regularization in Neural Network Models
• Modeling the origin of urban output scaling laws
• Time resolved 3Dinterferometric imaging of a section of a negative leader with LOFAR
• On a minimum distance procedure for threshold selection in tail analysis
• Assessment of long-range correlation in animal behaviour time series: the temporal pattern of locomotor activity of Japanese quail (Coturnix coturnix) and mosquito larva (Culex quinquefasciatus)
• On the extraction of power-law parts of the probability density functions in star-forming clouds
• Hypersparse Neural Network Analysis of Large-Scale Internet Traffic
• New Phenomena in Large-Scale Internet Traffic
• The Core Mass Function Across Galactic Environments. III. Massive Protoclusters
• The Energy Landscape of Neurophysiological Activity Implicit in Brain Network Structure
• Stylized facts in Brazilian vote distributions
• Survival Function Analysis of Planet Size Distribution with GAIA Data Release 2 Updates
• An evolving model for the supply network in a tourism destination
• Measuring the Complexity of Continuous Distributions
• A Quantitative Portrait of Wikipedia's High-Tempo Collaborations during the 2020 Coronavirus Pandemic
• Asymptotic independence and support detection techniques for heavy-tailed multivariate data
• Efficient Minimum Distance Estimation of Pareto Exponent from Top Income Shares
• Gravitationally Bound Gas Determines Star Formation in the Galaxy
• On the survival of the quantum depletion of a condensate after release from a magnetic trap
• Spectral Power-law Formation by Sequential Particle Acceleration in Multiple Flare Magnetic Islands
• Dynamics and potential origins of decimeter-sized particles around comet 67P/Churyumov-Gerasimenko
• Extraction of a second power-law tail of the density distribution in simulated clouds
• A complete data frame work for fitting power law distributions
• Tunable robustness in power-law inference
• Corrupted bifractal features in finite uncorrelated power-law distributed data
• Are human interactivity times lognormal?
• Motifs in earthquake networks: Romania, Italy, United States of America, and Japan
• Markov Chain Monte Carlo for generating ranked textual data
• Mal-Netminer: Malware Classification Approach based on Social Network Analysis of System Call Graph
• Multi-spatial Scale Event Detection from Geo-tagged Tweet Streams via Power-law Verification
• Degree-based Outlier Detection within IP Traffic Modelled as a Link Stream
• Revealing Task Driven Knowledge Worker Behaviors in Open Source Software Communities
• Nonparametric estimation of the preferential attachment function from one network snapshot
• Waiting times for sea level variations in the Port of Trieste: a computational data-driven study
• Image Analysis Enhanced Event Detection from Geo-tagged Tweet Streams
• Estimating the Total Volume of Queries to a Search Engine