Federated Continual Learning: Concepts, Challenges, and Solutions
arXiv:2502.07059 · doi:10.1016/j.neucom.2025.130844
Abstract
Federated Continual Learning (FCL) has emerged as a robust solution for collaborative model training in dynamic environments, where data samples are continuously generated and distributed across multiple devices. This survey provides a comprehensive review of FCL, focusing on key challenges such as heterogeneity, model stability, communication overhead, and privacy preservation. We explore various forms of heterogeneity and their impact on model performance. Solutions to non-IID data, resource-constrained platforms, and personalized learning are reviewed in an effort to show the complexities of handling heterogeneous data distributions. Next, we review techniques for ensuring model stability and avoiding catastrophic forgetting, which are critical in non-stationary environments. Privacy-preserving techniques are another aspect of FCL that have been reviewed in this work. This survey has integrated insights from federated learning and continual learning to present strategies for improving the efficacy and scalability of FCL systems, making it applicable to a wide range of real-world scenarios.
Accepted for publication in Neurocomputing
References in corpus (31)
- Overcoming catastrophic forgetting in neural networks
- Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
- Federated Learning with Non-IID Data
- A continual learning survey: Defying forgetting in classification tasks
- Towards Personalized Federated Learning
- Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence
- Three scenarios for continual learning
- A Survey on Metric Learning for Feature Vectors and Structured Data
- Improving Federated Learning Personalization via Model Agnostic Meta Learning
- Personalized Federated Learning: A Meta-Learning Approach
- On Tiny Episodic Memories in Continual Learning
- Federated Collaborative Filtering for Privacy-Preserving Personalized Recommendation System
- Decentralized Federated Learning: A Survey on Security and Privacy
- CINIC-10 is not ImageNet or CIFAR-10
- Federated Adversarial Domain Adaptation
- Federated Continual Learning via Knowledge Fusion: A Survey
- Abnormal Client Behavior Detection in Federated Learning
- Reservoir of Diverse Adaptive Learners and Stacking Fast Hoeffding Drift Detection Methods for Evolving Data Streams
- SlowMo: Improving Communication-Efficient Distributed SGD with Slow Momentum
- A distillation-based approach integrating continual learning and federated learning for pervasive services
- SocNav1: A Dataset to Benchmark and Learn Social Navigation Conventions
- DYNAMITE: Dynamic Interplay of Mini-Batch Size and Aggregation Frequency for Federated Learning with Static and Streaming Dataset
- AdaptCL: Adaptive Continual Learning for Tackling Heterogeneity in Sequential Datasets
- Self-supervised On-device Federated Learning from Unlabeled Streams
- Better Generative Replay for Continual Federated Learning
- How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Composition
- Tackling Dynamics in Federated Incremental Learning with Variational Embedding Rehearsal
- Federated Continual Learning for Edge-AI: A Comprehensive Survey
- Federated Learning with Regularized Client Participation
- Predictive Heterogeneity: Measures and Applications
- Privacy Amplification via Random Participation in Federated Learning