A survey on learning from imbalanced data streams: taxonomy, challenges, empirical study, and reproducible experimental framework

Abstract

Class imbalance poses new challenges when it comes to classifying data streams. Many algorithms recently proposed in the literature tackle this problem using a variety of data-level, algorithm-level, and ensemble approaches. However, there is a lack of standardized and agreed-upon procedures and benchmarks on how to evaluate these algorithms. This work proposes a standardized, exhaustive, and comprehensive experimental framework to evaluate algorithms in a collection of diverse and challenging imbalanced data stream scenarios. The experimental study evaluates 24 state-of-the-art data streams algorithms on 515 imbalanced data streams that combine static and dynamic class imbalance ratios, instance-level difficulties, concept drift, real-world and semi-synthetic datasets in binary and multi-class scenarios. This leads to a large-scale experimental study comparing state-of-the-art classifiers in the data stream mining domain. We discuss the advantages and disadvantages of state-of-the-art classifiers in each of these scenarios and we provide general recommendations to end-users for selecting the best algorithms for imbalanced data streams. Additionally, we formulate open challenges and future directions for this domain. Our experimental framework is fully reproducible and easy to extend with new methods. This way, we propose a standardized approach to conducting experiments in imbalanced data streams that can be used by other researchers to create complete, trustworthy, and fair evaluation of newly proposed methods. Our experimental framework can be downloaded from https://github.com/canoalberto/imbalanced-streams.

References in corpus (4)

• A Survey on Long-Tailed Visual Recognition
• Nonstationary data stream classification with online active learning and siamese neural networks
• Drift-Aware Multi-Memory Model for Imbalanced Data Streams
• Evaluating resampling methods on a real-life highly imbalanced online credit card payments dataset

Cited by in corpus (9)

• Active learning for data streams: a survey
• Resampling strategies for imbalanced regression: a survey and empirical analysis
• Nonstationary data stream classification with online active learning and siamese neural networks
• Class Balancing Diversity Multimodal Ensemble for Alzheimer's Disease Diagnosis and Early Detection
• Hoeffding adaptive trees for multi-label classification on data streams
• SiameseDuo++: Active Learning from Data Streams with Dual Augmented Siamese Networks
• AFS-BM: Enhancing Model Performance through Adaptive Feature Selection with Binary Masking
• Hybrid Imbalanced Regression Through Unified Data-Level and Algorithm-Level Balancing
• Structuring the Processing Frameworks for Data Stream Evaluation and Application