Problems with Fitting to the Power-Law Distribution

Abstract

This short communication uses a simple experiment to show that fitting to a power law distribution by using graphical methods based on linear fit on the log-log scale is biased and inaccurate. It shows that using maximum likelihood estimation (MLE) is far more robust. Finally, it presents a new table for performing the Kolmogorov-Smirnof test for goodness-of-fit tailored to power-law distributions in which the power-law exponent is estimated using MLE. The techniques presented here will advance the application of complex network theory by allowing reliable estimation of power-law models from data and further allowing quantitative assessment of goodness-of-fit of proposed power-law models to empirical data.

4 pages, 1 figure, 2 tables

Cited by in corpus (123)

• Power-law distributions in empirical data
• Power laws, Pareto distributions and Zipf's law
• Recommender Systems
• New Model of Internet Topology Using k-shell Decomposition
• MEDUSA - New Model of Internet Topology Using k-shell Decomposition
• Networks in Cognitive Science
• On the Frequency of Severe Terrorist Events
• Characterizing Human Mobility Patterns in a Large Street Network
• Power-law distributions in binned empirical data
• Optimal design of spatial distribution networks
• Observational constraints on the merger history of galaxies since $z\approx 6$: Probabilistic galaxy pair counts in the CANDELS fields
• Parameter estimation for power-law distributions by maximum likelihood methods
• Role of Activity in Human Dynamics
• Scale-free Networks Well Done
• Scale Invariance in Road Networks
• Zipf's Law Leads to Heaps' Law: Analyzing Their Relation in Finite-Size Systems
• Large-scale analysis of Zipf's law in English texts
• Empirical analysis of web-based user-object bipartite networks
• Maximum entropy principle and power-law tailed distributions
• Power-law size distributions in geoscience revisited
• Fitting heavy tailed distributions: the poweRlaw package
• Do wealth distributions follow power laws? Evidence from "rich lists"
• Self-exciting hurdle models for terrorist activity
• The Power-law Tail Exponent of Income Distributions
• Fitting Power-laws in empirical data with estimators that work for all exponents
• Statistical laws in linguistics
• Zipf Law for Brazilian Cities
• Social inertia in collaboration networks
• Emergence of scale-free leadership structure in social recommender systems
• Thresholded Power Law Size Distributions of Instabilities in Astrophysics
• On fitting the Pareto-Levy distribution to stock market index data: selecting a suitable cutoff value
• The Slope of the Source-Count Distribution for Fast Radio Bursts
• Flood avalanches in a semiarid basin with a dense reservoir network
• Analysis of power-law exponents by maximum-likelihood maps
• Scaling Laws in Human Language
• From Bitcoin to Bitcoin Cash: a network analysis
• Testing statistical laws in complex systems
• Extreme values and fat tails of multifractal fluctuations
• Evidence for the Gompertz Curve in the Income Distribution of Brazil 1978-2005
• Inverse cubic law of index fluctuation distribution in Indian markets
• Critical assessment of the evidence for striped nanoparticles
• Self-consistent method for density estimation
• Traditional and Heavy-Tailed Self Regularization in Neural Network Models
• Localizations on Complex Networks
• The Heavy-Tail Phenomenon in SGD
• Impact of Heterogeneous Human Activities on Epidemic Spreading
• Estimation of global network statistics from incomplete data
• The distinct flavors of Zipf's law in the rank-size and in the size-distribution representations, and its maximum-likelihood fitting
• Homogeneous temporal activity patterns in a large online communication space
• Autocorrelation analysis for the unbiased determination of power-law exponents in single-quantum-dot blinking
• Determination of the scale of coarse graining in earthquake network
• A Non-Gaussian Approach to Risk Measures
• Automated Swirl Detection Algorithm (ASDA) and Its Application to Simulation and Observational Data
• Long-range properties and data validity for hydrogeological time series: the case of the Paglia river
• Crackling to periodic dynamics in sheared granular media
• eBay users form stable groups of common interest
• Signatures of Lévy flights with annealed disorder
• A new catalogue of solar flare events from soft x-ray GOES signal in the period 1986-2020
• Empirical study on clique-degree distribution of networks
• Scaling Behaviors of Weighted Food Webs as Energy Transportation Networks
• Using a Power Law Distribution to describe Big Data
• What exactly are the properties of scale-free and other networks?
• Note on distribution free testing for discrete distributions
• A preferential attachment model with Poisson growth for scale-free networks
• Measuring Fundamental Properties of Real-World Complex Networks
• An Ultraviolet Survey of Low-Redshift Partial Lyman-Limit Systems with the HST Cosmic Origins Spectrograph
• Violation of the scaling relation and non-Markovian nature of earthquake aftershocks
• Acoustic emission across the magnetostructural transition of the giant magnetocaloric Gd5Si2Ge2 compound
• Scaling relation for earthquake networks
• Rank-frequency relation for Chinese characters
• Principle of Minimum Energy in Magnetic Reconnection in a Self-Organized Critical Model for Solar Flares
• The Anderson-Darling test of fit for the power law distribution from left censored samples
• Temporal analysis of acoustic emission from a plunged granular bed
• The Effect of Limited Sample Sizes on the Accuracy of the Estimated Scaling Parameter for Power-Law-Distributed Solar Data
• The Poissonian Origin of Power Laws in Solar Flare Waiting Time Distributions
• More for less: Predicting and maximizing genetic variant discovery via Bayesian nonparametrics
• The initial mass function modeled by a left truncated beta distribution
• The Solar Memory From Hours to Decades
• Simultaneous Observations of Giant Pulses from the Crab Pulsar, with the Murchison Widefield Array and Parkes Radio Telescope: Implications for the Giant Pulse Emission Mechanism
• Generalised Langevin Equation Formulation for Anomalous Diffusion in the Ising Model at the Critical Temperature
• Hurst exponent of very long birth time series in XX century Romania. Social and religious aspects
• Aftershocks in Modern Perspectives: Complex Earthquake Network, Aging, and Non-Markovianity
• 'It's fundamental': Quantum dot blinking experiment to teach critical thinking
• An Imaging Refractometer for Density Fluctuation Measurements in High Energy Density Plasmas
• Avalanche analysis from multi-electrode ensemble recordings in cat, monkey and human cerebral cortex during wakefulness and sleep
• Vulnerability of Networks Against Critical Link Failures
• Power Laws and the Cosmic Ray Energy Spectrum
• Non-parametric power-law surrogates
• On the survival of the quantum depletion of a condensate after release from a magnetic trap
• Comment on Yu et al., "High Quality Binary Protein Interaction Map of the Yeast Interactome Network." Science 322, 104 (2008)
• Divisibility patterns of natural numbers on a complex network
• Religion-based Urbanization Process in Italy: Statistical Evidence from Demographic and Economic Data
• An activity transition in FRB 20201124A: methodological rigor, detection of frequency-dependent cessation, and a geometric magnetar model
• The near-extreme density of intraday log-returns
• From Random Graph to Small World by Wandering
• Tunable robustness in power-law inference
• We cite as we communicate: A communication model for the citation process
• Frequently Co-cited Publications: Features and Kinetics
• In-Degree and PageRank of Web pages: Why do they follow similar power laws?
• Condition numbers and scale free graphs
• GB-KMV: An Augmented KMV Sketch for Approximate Containment Similarity Search
• Temperature dependence of dielectric permittivity in incommensurately modulated phase of ammonium fluoroberyllate
• Degree and connectivity of the Internet's scale-free topology
• Discrete Scale Invariance in Scale Free Graphs
• Evolutionary Trends of Developer Coordination: A Network Approach
• Modeling real spatial networks
• Complex Dependencies in Large Software Systems
• A Compounded Burr Probability Distribution for Fitting Heavy-Tailed Data with Applications to Biological Networks
• Zipf's Law for All the Natural Cities in the United States: A Geospatial Perspective
• Statistical Methods for Investigating the Cosmic Ray Energy Spectrum
• Multi-Avalanche Correlations in Directed Sandpile Models
• Cooperation Networks: Endogeneity and Complexity
• Statistical Tools for Analyzing the Cosmic Ray Energy Spectrum
• Tail Index Estimation for Discrete Heavy-Tailed Distributions
• Comparative analysis of transport communication networks and q-type statistics
• A note on some information-theoretic divergences between Zeta distributions
• On the Estimation and Use of Statistical Modelling in Information Retrieval
• Preferential attachment: a multi-attribute growth process generating scale-free networks of different topologies
• A Goodness-of-Fit Test for Sampled Subgraphs
• Evidence of Self-Organization in Time Series of Capital Markets
• Estimating the Total Volume of Queries to a Search Engine
• Modeling non-stationarities in high-frequency financial time series
• Statistical Analysis of Minifilament Eruptions Using Full-disk H$\alpha$ Blue-wing Observations at Big Bear Solar Observatory