Class-incremental Learning via Deep Model Consolidation

Abstract

Deep neural networks (DNNs) often suffer from "catastrophic forgetting" during incremental learning (IL) --- an abrupt degradation of performance on the original set of classes when the training objective is adapted to a newly added set of classes. Existing IL approaches tend to produce a model that is biased towards either the old classes or new classes, unless with the help of exemplars of the old data. To address this issue, we propose a class-incremental learning paradigm called Deep Model Consolidation (DMC), which works well even when the original training data is not available. The idea is to first train a separate model only for the new classes, and then combine the two individual models trained on data of two distinct set of classes (old classes and new classes) via a novel double distillation training objective. The two existing models are consolidated by exploiting publicly available unlabeled auxiliary data. This overcomes the potential difficulties due to the unavailability of original training data. Compared to the state-of-the-art techniques, DMC demonstrates significantly better performance in image classification (CIFAR-100 and CUB-200) and object detection (PASCAL VOC 2007) in the single-headed IL setting.

arXiv commentWACV 2020 camera-ready

References in corpus (16)

• Distilling the Knowledge in a Neural Network
• Densely Connected Convolutional Networks
• Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence
• Continual Learning Through Synaptic Intelligence
• An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks
• Fully Convolutional Networks for Semantic Segmentation
• Efficient Lifelong Learning with A-GEM
• Feature Pyramid Networks for Object Detection
• A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasets
• FearNet: Brain-Inspired Model for Incremental Learning
• Progress & Compress: A scalable framework for continual learning
• Overcoming Catastrophic Forgetting by Incremental Moment Matching
• COCO-Text: Dataset and Benchmark for Text Detection and Recognition in Natural Images
• Overcoming catastrophic forgetting with hard attention to the task
• Rotate your Networks: Better Weight Consolidation and Less Catastrophic Forgetting
• A Fully Convolutional Tri-branch Network (FCTN) for Domain Adaptation

Cited by in corpus (5)

• Maintaining Discrimination and Fairness in Class Incremental Learning
• An Adaptive Random Path Selection Approach for Incremental Learning
• Single-Net Continual Learning with Progressive Segmented Training (PST)
• Incremental Concept Learning via Online Generative Memory Recall
• Incremental Cross-Domain Adaptation for Robust Retinopathy Screening via Bayesian Deep Learning