GraphBLAS on the Edge: Anonymized High Performance Streaming of Network Traffic

Abstract

Long range detection is a cornerstone of defense in many operating domains (land, sea, undersea, air, space, ..,). In the cyber domain, long range detection requires the analysis of significant network traffic from a variety of observatories and outposts. Construction of anonymized hypersparse traffic matrices on edge network devices can be a key enabler by providing significant data compression in a rapidly analyzable format that protects privacy. GraphBLAS is ideally suited for both constructing and analyzing anonymized hypersparse traffic matrices. The performance of GraphBLAS on an Accolade Technologies edge network device is demonstrated on a near worse case traffic scenario using a continuous stream of CAIDA Telescope darknet packets. The performance for varying numbers of traffic buffers, threads, and processor cores is explored. Anonymized hypersparse traffic matrices can be constructed at a rate of over 50,000,000 packets per second; exceeding a typical 400 Gigabit network link. This performance demonstrates that anonymized hypersparse traffic matrices are readily computable on edge network devices with minimal compute resources and can be a viable data product for such devices.

Accepted to IEEE HPEC, Outstanding Paper Award, 8 pages, 8 figures, 1 table, 70 references. arXiv admin note: text overlap with arXiv:2108.06653, arXiv:2008.00307, arXiv:2203.10230

References in corpus (11)

• Community Structure in Time-Dependent, Multiscale, and Multiplex Networks
• Graphs, Matrices, and the GraphBLAS: Seven Good Reasons
• Enabling Massive Deep Neural Networks with the GraphBLAS
• RedisGraph GraphBLAS Enabled Graph Database
• Hypersparse Neural Network Analysis of Large-Scale Internet Traffic
• Streaming 1.9 Billion Hypersparse Network Updates per Second with D4M
• Spatial Temporal Analysis of 40,000,000,000,000 Internet Darkspace Packets
• 75,000,000,000 Streaming Inserts/Second Using Hierarchical Hypersparse GraphBLAS Matrices
• Multi-Temporal Analysis and Scaling Relations of 100,000,000,000 Network Packets
• Zero Botnets: An Observe-Pursue-Counter Approach
• Hybrid Power-Law Models of Network Traffic

Cited by in corpus (5)

• Anonymized Network Sensing Graph Challenge
• Deployment of Real-Time Network Traffic Analysis using GraphBLAS Hypersparse Matrices and D4M Associative Arrays
• Extracting TCPIP Headers at High Speed for the Anonymized Network Traffic Graph Challenge
• Focusing and Calibration of Large Scale Network Sensors using GraphBLAS Anonymized Hypersparse Matrices
• DBOS Network Sensing: A Web Services Approach to Collaborative Awareness