Spatial Temporal Analysis of 40,000,000,000,000 Internet Darkspace Packets

Abstract

The Internet has never been more important to our society, and understanding the behavior of the Internet is essential. The Center for Applied Internet Data Analysis (CAIDA) Telescope observes a continuous stream of packets from an unsolicited darkspace representing 1/256 of the Internet. During 2019 and 2020 over 40,000,000,000,000 unique packets were collected representing the largest ever assembled public corpus of Internet traffic. Using the combined resources of the Supercomputing Centers at UC San Diego, Lawrence Berkeley National Laboratory, and MIT, the spatial temporal structure of anonymized source-destination pairs from the CAIDA Telescope data has been analyzed with GraphBLAS hierarchical hypersparse matrices. These analyses provide unique insight on this unsolicited Internet darkspace traffic with the discovery of many previously unseen scaling relations. The data show a significant sustained increase in unsolicited traffic corresponding to the start of the COVID19 pandemic, but relatively little change in the underlying scaling relations associated with unique sources, source fan-outs, unique links, destination fan-ins, and unique destinations. This work provides a demonstration of the practical feasibility and benefit of the safe collection and analysis of significant quantities of anonymized Internet traffic.

8 pages, 9 figures, 2 tables, 43 references, accepted to IEEE HPEC 2021. arXiv admin note: substantial text overlap with arXiv:2008.00307

References in corpus (13)

• Emergence of scaling in random networks
• Power-law distributions in empirical data
• The diameter of the world wide web
• Community Structure in Time-Dependent, Multiscale, and Multiplex Networks
• Interactive Supercomputing on 40,000 Cores for Machine Learning and Data Analysis
• Hypersparse Neural Network Analysis of Large-Scale Internet Traffic
• Streaming 1.9 Billion Hypersparse Network Updates per Second with D4M
• 75,000,000,000 Streaming Inserts/Second Using Hierarchical Hypersparse GraphBLAS Matrices
• Multi-Temporal Analysis and Scaling Relations of 100,000,000,000 Network Packets
• Zero Botnets: An Observe-Pursue-Counter Approach
• Large Scale Parallelization Using File-Based Communications
• Optimizing Xeon Phi for Interactive Data Analysis
• Hybrid Power-Law Models of Network Traffic

Cited by in corpus (7)

• GraphBLAS on the Edge: Anonymized High Performance Streaming of Network Traffic
• Large Scale Enrichment and Statistical Cyber Characterization of Network Traffic (Enriquecimiento a gran escala y caracterización cibernética estadística del tráfico de red)
• Deployment of Real-Time Network Traffic Analysis using GraphBLAS Hypersparse Matrices and D4M Associative Arrays
• Temporal Correlation of Internet Observatories and Outposts
• Focusing and Calibration of Large Scale Network Sensors using GraphBLAS Anonymized Hypersparse Matrices
• Teaching Network Traffic Matrices in an Interactive Game Environment
• What is Normal? A Big Data Observational Science Model of Anonymized Internet Traffic