Multi-Temporal Analysis and Scaling Relations of 100,000,000,000 Network Packets

Abstract

Our society has never been more dependent on computer networks. Effective utilization of networks requires a detailed understanding of the normal background behaviors of network traffic. Large-scale measurements of networks are computationally challenging. Building on prior work in interactive supercomputing and GraphBLAS hypersparse hierarchical traffic matrices, we have developed an efficient method for computing a wide variety of streaming network quantities on diverse time scales. Applying these methods to 100,000,000,000 anonymized source-destination pairs collected at a network gateway reveals many previously unobserved scaling relationships. These observations provide new insights into normal network background traffic that could be used for anomaly detection, AI feature engineering, and testing theoretical models of streaming networks.

6 pages, 6 figures,3 tables, 49 references, accepted to IEEE HPEC 2020

References in corpus (2)

• Power-law distributions in empirical data
• Navigability of Complex Networks

Cited by in corpus (9)

• Spatial Temporal Analysis of 40,000,000,000,000 Internet Darkspace Packets
• GraphBLAS on the Edge: Anonymized High Performance Streaming of Network Traffic
• Anonymized Network Sensing Graph Challenge
• Deployment of Real-Time Network Traffic Analysis using GraphBLAS Hypersparse Matrices and D4M Associative Arrays
• Temporal Correlation of Internet Observatories and Outposts
• Vertical, Temporal, and Horizontal Scaling of Hierarchical Hypersparse GraphBLAS Matrices
• Focusing and Calibration of Large Scale Network Sensors using GraphBLAS Anonymized Hypersparse Matrices
• What is Normal? A Big Data Observational Science Model of Anonymized Internet Traffic
• Teaching Network Traffic Matrices in an Interactive Game Environment