Rainbow Memory: Continual Learning with a Memory of Diverse Samples
arXiv:2103.17230
Abstract
Continual learning is a realistic learning scenario for AI models. Prevalent scenario of continual learning, however, assumes disjoint sets of classes as tasks and is less realistic rather artificial. Instead, we focus on 'blurry' task boundary; where tasks shares classes and is more realistic and practical. To address such task, we argue the importance of diversity of samples in an episodic memory. To enhance the sample diversity in the memory, we propose a novel memory management strategy based on per-sample classification uncertainty and data augmentation, named Rainbow Memory (RM). With extensive empirical validations on MNIST, CIFAR10, CIFAR100, and ImageNet datasets, we show that the proposed method significantly improves the accuracy in blurry continual learning setups, outperforming state of the arts by large margins despite its simplicity. Code and data splits will be available in https://github.com/clovaai/rainbow-memory.
Accepted paper at CVPR 2021
References in corpus (13)
- Overcoming catastrophic forgetting in neural networks
- mixup: Beyond Empirical Risk Minimization
- Improved Regularization of Convolutional Neural Networks with Cutout
- Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence
- Continual Learning Through Synaptic Intelligence
- An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks
- Overcoming Catastrophic Forgetting by Incremental Moment Matching
- Parameter-Efficient Transfer Learning for NLP
- NSML: Meet the MLaaS platform with a real-world case study
- Continual Learning in Generative Adversarial Nets
- Coresets via Bilevel Optimization for Continual Learning and Streaming
- Memory Replay GANs: learning to generate images from new categories without forgetting
- The Effectiveness of Memory Replay in Large Scale Continual Learning