Federated Continual Learning: Concepts, Challenges, and Solutions

Abstract

Federated Continual Learning (FCL) has emerged as a robust solution for collaborative model training in dynamic environments, where data samples are continuously generated and distributed across multiple devices. This survey provides a comprehensive review of FCL, focusing on key challenges such as heterogeneity, model stability, communication overhead, and privacy preservation. We explore various forms of heterogeneity and their impact on model performance. Solutions to non-IID data, resource-constrained platforms, and personalized learning are reviewed in an effort to show the complexities of handling heterogeneous data distributions. Next, we review techniques for ensuring model stability and avoiding catastrophic forgetting, which are critical in non-stationary environments. Privacy-preserving techniques are another aspect of FCL that have been reviewed in this work. This survey has integrated insights from federated learning and continual learning to present strategies for improving the efficacy and scalability of FCL systems, making it applicable to a wide range of real-world scenarios.

Accepted for publication in Neurocomputing

References in corpus (19)

• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Three scenarios for continual learning
• On Tiny Episodic Memories in Continual Learning
• Federated Collaborative Filtering for Privacy-Preserving Personalized Recommendation System
• Decentralized Federated Learning: A Survey on Security and Privacy
• Federated Continual Learning via Knowledge Fusion: A Survey
• Abnormal Client Behavior Detection in Federated Learning
• SlowMo: Improving Communication-Efficient Distributed SGD with Slow Momentum
• A distillation-based approach integrating continual learning and federated learning for pervasive services
• DYNAMITE: Dynamic Interplay of Mini-Batch Size and Aggregation Frequency for Federated Learning with Static and Streaming Dataset
• AdaptCL: Adaptive Continual Learning for Tackling Heterogeneity in Sequential Datasets
• Self-supervised On-device Federated Learning from Unlabeled Streams
• Better Generative Replay for Continual Federated Learning
• How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Composition
• Tackling Dynamics in Federated Incremental Learning with Variational Embedding Rehearsal
• Federated Continual Learning for Edge-AI: A Comprehensive Survey
• Federated Learning with Regularized Client Participation
• Predictive Heterogeneity: Measures and Applications
• Privacy Amplification via Random Participation in Federated Learning

Cited by in corpus (1)

• Federated continual learning: A comprehensive survey on lifelong and privacy-preserving learning over distributed and non-stationary data