Detection and localization of change-points in high-dimensional network traffic data

Abstract

We propose a novel and efficient method, that we shall call TopRank in the following paper, for detecting change-points in high-dimensional data. This issue is of growing concern to the network security community since network anomalies such as Denial of Service (DoS) attacks lead to changes in Internet traffic. Our method consists of a data reduction stage based on record filtering, followed by a nonparametric change-point detection test based on $U$-statistics. Using this approach, we can address massive data streams and perform anomaly detection and localization on the fly. We show how it applies to some real Internet traffic provided by France-Télécom (a French Internet service provider) in the framework of the ANR-RNRT OSCAR project. This approach is very attractive since it benefits from a low computational load and is able to detect and localize several types of network anomalies. We also assess the performance of the TopRank algorithm using synthetic data and compare it with alternative approaches based on random aggregation.

Published in at http://dx.doi.org/10.1214/08-AOAS232 the Annals of Applied Statistics (http://www.imstat.org/aoas/) by the Institute of Mathematical Statistics (http://www.imstat.org)

Cited by in corpus (14)

• Selective review of offline change point detection methods
• Sequential multi-sensor change-point detection
• Uniform change point tests in high dimension
• The group fused Lasso for multiple change-point detection
• Homogeneity and change-point detection tests for multivariate data using rank statistics
• Distributed detection/localization of change-points in high-dimensional network traffic data
• Sequential Change Detection in the Presence of Unknown Parameters
• Accounting for stellar activity signals in radial-velocity data by using Change Point Detection techniques
• Consistent detection and optimal localization of all detectable change points in piecewise stationary arbitrarily sparse network-sequences
• Pesticide concentration monitoring: investigating spatio-temporal patterns in left censored data
• Valid and Exact Statistical Inference for Multi-dimensional Multiple Change-Points by Selective Inference
• Locally private online change point detection
• Kernel Clustering with Sigmoid-based Regularization for Efficient Segmentation of Sequential Data
• MT4G: A Tool for Reliable Auto-Discovery of NVIDIA and AMD GPU Compute and Memory Topologies