Generative Feature Replay For Class-Incremental Learning
arXiv:2004.09199
Abstract
Humans are capable of learning new tasks without forgetting previous ones, while neural networks fail due to catastrophic forgetting between new and previously-learned tasks. We consider a class-incremental setting which means that the task-ID is unknown at inference time. The imbalance between old and new classes typically results in a bias of the network towards the newest ones. This imbalance problem can either be addressed by storing exemplars from previous tasks, or by using image replay methods. However, the latter can only be applied to toy datasets since image generation for complex datasets is a hard problem. We propose a solution to the imbalance problem based on generative feature replay which does not require any exemplars. To do this, we split the network into two parts: a feature extractor and a classifier. To prevent forgetting, we combine generative feature replay in the classifier with feature distillation in the feature extractor. Through feature generation, our method reduces the complexity of generative replay and prevents the imbalance problem. Our approach is computationally efficient and scalable to large datasets. Experiments confirm that our approach achieves state-of-the-art results on CIFAR-100 and ImageNet, while requiring only a fraction of the storage needed for exemplar-based continual learning. Code available at \url{https://github.com/xialeiliu/GFR-IL}.
Accepted at CVPR2020: Workshop on Continual Learning in Computer Vision
References in corpus (18)
- Overcoming catastrophic forgetting in neural networks
- Large Scale GAN Training for High Fidelity Natural Image Synthesis
- Generative Adversarial Text to Image Synthesis
- A continual learning survey: Defying forgetting in classification tasks
- Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence
- Continual Learning Through Synaptic Intelligence
- A Learned Representation For Artistic Style
- Learning to Discover Cross-Domain Relations with Generative Adversarial Networks
- Invertible Conditional GANs for image editing
- Efficient Lifelong Learning with A-GEM
- Three scenarios for continual learning
- cGANs with Projection Discriminator
- FearNet: Brain-Inspired Model for Incremental Learning
- SVCCA: Singular Vector Canonical Correlation Analysis for Deep Learning Dynamics and Interpretability
- Don't forget, there is more than forgetting: new metrics for Continual Learning
- Are GANs Created Equal? A Large-Scale Study
- Memory Replay GANs: learning to generate images from new categories without forgetting
- Class-Splitting Generative Adversarial Networks