Continual Learning in the Presence of Repetition
arXiv:2405.04101 · doi:10.1016/j.neunet.2024.106920
Abstract
Continual learning (CL) provides a framework for training models in ever-evolving environments. Although re-occurrence of previously seen objects or tasks is common in real-world problems, the concept of repetition in the data stream is not often considered in standard benchmarks for CL. Unlike with the rehearsal mechanism in buffer-based strategies, where sample repetition is controlled by the strategy, repetition in the data stream naturally stems from the environment. This report provides a summary of the CLVision challenge at CVPR 2023, which focused on the topic of repetition in class-incremental learning. The report initially outlines the challenge objective and then describes three solutions proposed by finalist teams that aim to effectively exploit the repetition in the stream to learn continually. The experimental results from the challenge highlight the effectiveness of ensemble-based solutions that employ multiple versions of similar modules, each trained on different but overlapping subsets of classes. This report underscores the transformative potential of taking a different perspective in CL by employing repetition in the data stream to foster innovative strategy design.
Accepted version, to appear in Neural Networks; Challenge Report of the 4th Workshop on Continual Learning in Computer Vision at CVPR
References in corpus (12)
- PathNet: Evolution Channels Gradient Descent in Super Neural Networks
- AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty
- On Tiny Episodic Memories in Continual Learning
- Incorporating Neuro-Inspired Adaptability for Continual Learning in Artificial Intelligence
- Open-Set Recognition: a Good Closed-Set Classifier is All You Need?
- Continual Learning: Applications and the Road Forward
- Avalanche: A PyTorch Library for Deep Continual Learning
- Divide and not forget: Ensemble of selectively trained experts in Continual Learning
- 3rd Continual Learning Workshop Challenge on Egocentric Category and Instance Level Object Understanding
- Neuromimetic metaplasticity for adaptive continual learning
- HAT-CL: A Hard-Attention-to-the-Task PyTorch Library for Continual Learning
- Knowledge Accumulation in Continually Learned Representations and the Issue of Feature Forgetting