End-to-End Incremental Learning

Abstract

Although deep learning approaches have stood out in recent years due to their state-of-the-art results, they continue to suffer from catastrophic forgetting, a dramatic decrease in overall performance when training with new classes added incrementally. This is due to current neural network architectures requiring the entire dataset, consisting of all the samples from the old as well as the new classes, to update the model -a requirement that becomes easily unsustainable as the number of classes grows. We address this issue with our approach to learn deep neural networks incrementally, using new data and only a small exemplar set corresponding to samples from the old classes. This is based on a loss composed of a distillation measure to retain the knowledge acquired from the old classes, and a cross-entropy loss to learn the new classes. Our incremental training is achieved while keeping the entire framework end-to-end, i.e., learning the data representation and the classifier jointly, unlike recent methods with no such guarantees. We evaluate our method extensively on the CIFAR-100 and ImageNet (ILSVRC 2012) image classification datasets, and show state-of-the-art performance.

arXiv commentTo appear in ECCV 2018

References in corpus (6)

• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• Distilling the Knowledge in a Neural Network
• Overcoming catastrophic forgetting in neural networks
• Two-Stream Convolutional Networks for Action Recognition in Videos
• An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks
• Gradient Episodic Memory for Continual Learning

Cited by in corpus (13)

• A Comprehensive Study of Class Incremental Learning Algorithms for Visual Tasks
• RATT: Recurrent Attention to Transient Tasks for Continual Image Captioning
• Semantic Drift Compensation for Class-Incremental Learning
• ACE: Adapting to Changing Environments for Semantic Segmentation
• ConDA: Continual Unsupervised Domain Adaptation
• BI-MAML: Balanced Incremental Approach for Meta Learning
• Layerwise Optimization by Gradient Decomposition for Continual Learning
• Adaptive scheduling for adaptive sampling in POS taggers construction
• SID: Incremental Learning for Anchor-Free Object Detection via Selective and Inter-Related Distillation
• RSAC: Regularized Subspace Approximation Classifier for Lightweight Continuous Learning
• Online Continual Learning in Image Classification: An Empirical Survey
• Adversarial Incremental Learning
• Discriminative Distillation to Reduce Class Confusion in Continual Learning