New Phenomena in Large-Scale Internet Traffic
arXiv:2201.06096 · doi:10.1201/9781003033707
Abstract
The Internet is transforming our society, necessitating a quantitative understanding of Internet traffic. Our team collects and curates the largest publicly available Internet traffic data sets. An analysis of 50 billion packets using 10,000 processors in the MIT SuperCloud reveals a new phenomenon: the importance of otherwise unseen leaf nodes and isolated links in Internet traffic. Our analysis further shows that a two-parameter modified Zipf-Mandelbrot distribution accurately describes a wide variety of source/destination statistics on moving sample windows ranging from 100{,}000 to 100{,}000{,}000 packets over collections that span years and continents. The measured model parameters distinguish different network streams, and the model leaf parameter strongly correlates with the fraction of the traffic in different underlying network topologies.
arXiv comment53 pages, 27 figures, 8 tables, 121 references. Portions of this work originally appeared as arXiv:1904.04396v1 which has been split for publication in the book "Massive Graph Analytics" (edited by David Bader)
References in corpus (22)
- Emergence of scaling in random networks
- Statistical mechanics of complex networks
- Identification of influential spreaders in complex networks
- Community Structure in Time-Dependent, Multiscale, and Multiplex Networks
- Clustering and preferential attachment in growing networks
- Higher-order organization of complex networks
- Hyperbolic Geometry of Complex Networks
- Control Principles of Complex Networks
- Popularity versus Similarity in Growing Networks
- The fundamental advantages of temporal networks
- Navigability of Complex Networks
- Sustaining the Internet with Hyperbolic Mapping
- Beyond the Zipf-Mandelbrot law in quantitative linguistics
- Power-law distributions in binned empirical data
- Interactive Supercomputing on 40,000 Cores for Machine Learning and Data Analysis
- Curvature and temperature of complex networks
- Minimum energy control for complex networks
- A Preferential Attachment Paradox: How Preferential Attachment Combines with Growth to Produce Networks with Log-normal In-degree Distributions
- Navigating ultrasmall worlds in ultrashort time
- Long-Range Correlations and Memory in the Dynamics of Internet Interdomain Routing
- A Scale Invariant Approach for Sparse Signal Recovery
- Scaling in Internet Traffic: a 14 year and 3 day longitudinal study, with multiscale analyses and random projections