Mnemonics Training: Multi-Class Incremental Learning without Forgetting

Abstract

Multi-Class Incremental Learning (MCIL) aims to learn new concepts by incrementally updating a model trained on previous concepts. However, there is an inherent trade-off to effectively learning new concepts without catastrophic forgetting of previous ones. To alleviate this issue, it has been proposed to keep around a few examples of the previous concepts but the effectiveness of this approach heavily depends on the representativeness of these examples. This paper proposes a novel and automatic framework we call mnemonics, where we parameterize exemplars and make them optimizable in an end-to-end manner. We train the framework through bilevel optimizations, i.e., model-level and exemplar-level. We conduct extensive experiments on three MCIL benchmarks, CIFAR-100, ImageNet-Subset and ImageNet, and show that using mnemonics exemplars can surpass the state-of-the-art by a large margin. Interestingly and quite intriguingly, the mnemonics exemplars tend to be on the boundaries between different classes.

arXiv commentExperiment results updated (different from the conference version). Code is available at https://github.com/yaoyao-liu/mnemonics-training

References in corpus (1)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library

Cited by in corpus (13)

• Class-Incremental Learning: A Survey
• Adaptive Aggregation Networks for Class-Incremental Learning
• An Ensemble of Epoch-wise Empirical Bayes for Few-shot Learning
• Learning without Forgetting for Vision-Language Models
• Unified Probabilistic Deep Continual Learning through Generative Replay and Open Set Recognition
• Class-incremental Learning with Pre-allocated Fixed Classifiers
• An Investigation of Replay-based Approaches for Continual Learning
• Multi-Granularity Regularized Re-Balancing for Class Incremental Learning
• Balanced softmax cross-entropy for incremental learning with and without memory
• Unveiling the Tapestry: the Interplay of Generalization and Forgetting in Continual Learning
• Tuned Compositional Feature Replays for Efficient Stream Learning
• Improving Forward Compatibility in Class Incremental Learning by Increasing Representation Rank and Feature Richness
• EXACFS -- A CIL Method to mitigate Catastrophic Forgetting