A Lightweight Concept Drift Detection and Adaptation Framework for IoT Data Streams

Abstract

In recent years, with the increasing popularity of "Smart Technology", the number of Internet of Things (IoT) devices and systems have surged significantly. Various IoT services and functionalities are based on the analytics of IoT streaming data. However, IoT data analytics faces concept drift challenges due to the dynamic nature of IoT systems and the ever-changing patterns of IoT data streams. In this article, we propose an adaptive IoT streaming data analytics framework for anomaly detection use cases based on optimized LightGBM and concept drift adaptation. A novel drift adaptation method named Optimized Adaptive and Sliding Windowing (OASW) is proposed to adapt to the pattern changes of online IoT data streams. Experiments on two public datasets show the high accuracy and efficiency of our proposed adaptive LightGBM model compared against other state-of-the-art approaches. The proposed adaptive LightGBM model can perform continuous learning and drift adaptation on IoT data streams without human intervention.

arXiv commentAccepted and to appear in IEEE Internet of Things Magazine; Code is available at Github link:https://github.com/Western-OC2-Lab/OASW-Concept-Drift-Detection-and-Adaptation

References in corpus (3)

• On Hyperparameter Optimization of Machine Learning Algorithms: Theory and Practice
• Learning under Concept Drift: A Review
• Tree-based Intelligent Intrusion Detection System in Internet of Vehicles

Cited by in corpus (12)

• MTH-IDS: A Multi-Tiered Hybrid Intrusion Detection System for Internet of Vehicles
• A Transfer Learning and Optimized CNN Based Intrusion Detection System for Internet of Vehicles
• IoT Data Analytics in Dynamic Environments: From An Automated Machine Learning Perspective
• PWPAE: An Ensemble Framework for Concept Drift Adaptation in IoT Data Streams
• Multi-Perspective Content Delivery Networks Security Framework Using Optimized Unsupervised Anomaly Detection
• Concept Drift Detection in Federated Networked Systems
• Enabling AutoML for Zero-Touch Network Security: Use-Case Driven Analysis
• Towards Zero Touch Networks: Cross-Layer Automated Security Solutions for 6G Wireless Networks
• A Model Drift Detection and Adaptation Framework for 5G Core Networks
• A Transfer Learning Framework for Anomaly Detection in Multivariate IoT Traffic Data
• An Edge-Cloud Integrated Framework for Flexible and Dynamic Stream Analytics
• Addressing Data Distribution Shifts in Online Machine Learning Powered Smart City Applications Using Augmented Test-Time Adaptation