Characterizing Concept Drift

Abstract

Most machine learning models are static, but the world is dynamic, and increasing online deployment of learned models gives increasing urgency to the development of efficient and effective mechanisms to address learning in the context of non-stationary distributions, or as it is commonly called concept drift. However, the key issue of characterizing the different types of drift that can occur has not previously been subjected to rigorous definition and analysis. In particular, while some qualitative drift categorizations have been proposed, few have been formally defined, and the quantitative descriptions required for precise and objective understanding of learner performance have not existed. We present the first comprehensive framework for quantitative analysis of drift. This supports the development of the first comprehensive set of formal definitions of types of concept drift. The formal definitions clarify ambiguities and identify gaps in previous definitions, giving rise to a new comprehensive taxonomy of concept drift types and a solid foundation for research into mechanisms to detect and address concept drift.

Accepted for publication in Data Mining and Knowledge Discovery

References in corpus (1)

• Learning under Concept Drift: an Overview

Cited by in corpus (57)

• A survey on learning from imbalanced data streams: taxonomy, challenges, empirical study, and reproducible experimental framework
• Correlation-aware Spatial-Temporal Graph Learning for Multivariate Time-series Anomaly Detection
• Streaming Graph Neural Networks via Continual Learning
• Adaptation Strategies for Automated Machine Learning on Evolving Data
• Concept drift detection and adaptation for federated and continual learning
• Persistence in Complex Systems
• Online Learning With Adaptive Rebalancing in Nonstationary Environments
• Online AutoML: An adaptive AutoML framework for online learning
• Diagnosing Concept Drift with Visual Analytics
• A Classification of Feedback Loops and Their Relation to Biases in Automated Decision-Making Systems
• Lifelong Self-Adaptation: Self-Adaptation Meets Lifelong Machine Learning
• Learning Parameter Distributions to Detect Concept Drift in Data Streams
• Lifelong Learning of Graph Neural Networks for Open-World Node Classification
• Bridging the gap: Towards an Expanded Toolkit for AI-driven Decision-Making in the Public Sector
• WATCH: Wasserstein Change Point Detection for High-Dimensional Time Series Data
• Dealing with Drift of Adaptation Spaces in Learning-based Self-Adaptive Systems using Lifelong Self-Adaptation
• Lifelong Learning on Evolving Graphs Under the Constraints of Imbalanced Classes and New Classes
• Empirically Measuring Transfer Distance for System Design and Operation
• Handling Concept Drift for Predictions in Business Process Mining
• Fraud Analytics: A Decade of Research -- Organizing Challenges and Solutions in the Field
• Addressing machine learning concept drift reveals declining vaccine sentiment during the COVID-19 pandemic
• Using AntiPatterns to avoid MLOps Mistakes
• Performative Prediction
• Analyzing Flight Delay Prediction Under Concept Drift
• Online optimal exact identification of a quantum change point
• Learning from Data Streams: An Overview and Update
• Streaming Active Deep Forest for Evolving Data Stream Classification
• Incremental Learning In Online Scenario
• Towards Identifying and Managing Sources of Uncertainty in AI and Machine Learning Models - An Overview
• DeepObliviate: A Powerful Charm for Erasing Data Residual Memory in Deep Neural Networks
• Engineering problems in machine learning systems
• A Probabilistic Framework for Adapting to Changing and Recurring Concepts in Data Streams
• Concept Drift Learning with Alternating Learners
• On the Costs and Benefits of Adopting Lifelong Learning for Software Analytics -- Empirical Study on Brown Build and Risk Prediction
• Power Plant Performance Modeling with Concept Drift
• More Than Meets The Eye: Semi-supervised Learning Under Non-IID Data
• Adaptive Fraud Detection System Using Dynamic Risk Features
• Deep Unsupervised Domain Adaptation for Time Series Classification: a Benchmark
• Learning Football Body-Orientation as a Matter of Classification
• Labelling Drifts in a Fault Detection System for Wind Turbine Maintenance
• A Conceptual Framework for Lifelong Learning
• On the Inter-relationships among Drift rate, Forgetting rate, Bias/variance profile and Error
• Emergent and Unspecified Behaviors in Streaming Decision Trees
• Sketches for Time-Dependent Machine Learning
• DAO-GP Drift Aware Online Non-Linear Regression Gaussian-Process
• DSAP: Analyzing Bias Through Demographic Comparison of Datasets
• How to Sustainably Monitor ML-Enabled Systems? Accuracy and Energy Efficiency Tradeoffs in Concept Drift Detection
• A Self-Organizing Clustering System for Unsupervised Distribution Shift Detection
• Ranking and benchmarking framework for sampling algorithms on synthetic data streams
• Scalable Teacher Forcing Network for Semi-Supervised Large Scale Data Streams
• Contextual One-Class Classification in Data Streams
• A Scalable Approach to Covariate and Concept Drift Management via Adaptive Data Segmentation
• An Eager Splitting Strategy for Online Decision Trees
• Describing Nonstationary Data Streams in Frequency Domain
• Drift Estimation with Graphical Models
• Tackling Virtual and Real Concept Drifts: An Adaptive Gaussian Mixture Model
• A Practical Guide to Streaming Continual Learning