PKI: Prior Knowledge-Infused Neural Network for Few-Shot Class-Incremental Learning

Abstract

Few-shot class-incremental learning (FSCIL) aims to continually adapt a model on a limited number of new-class examples, facing two well-known challenges: catastrophic forgetting and overfitting to new classes. Existing methods tend to freeze more parts of network components and finetune others with an extra memory during incremental sessions. These methods emphasize preserving prior knowledge to ensure proficiency in recognizing old classes, thereby mitigating catastrophic forgetting. Meanwhile, constraining fewer parameters can help in overcoming overfitting with the assistance of prior knowledge. Following previous methods, we retain more prior knowledge and propose a prior knowledge-infused neural network (PKI) to facilitate FSCIL. PKI consists of a backbone, an ensemble of projectors, a classifier, and an extra memory. In each incremental session, we build a new projector and add it to the ensemble. Subsequently, we finetune the new projector and the classifier jointly with other frozen network components, ensuring the rich prior knowledge is utilized effectively. By cascading projectors, PKI integrates prior knowledge accumulated from previous sessions and learns new knowledge flexibly, which helps to recognize old classes and efficiently learn new classes. Further, to reduce the resource consumption associated with keeping many projectors, we design two variants of the prior knowledge-infused neural network (PKIV-1 and PKIV-2) to trade off a balance between resource consumption and performance by reducing the number of projectors. Extensive experiments on three popular benchmarks demonstrate that our approach outperforms state-of-the-art methods.

References in corpus (10)

• Overcoming catastrophic forgetting in neural networks
• Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence
• Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time
• Memorizing Complementation Network for Few-Shot Class-Incremental Learning
• Memory Replay GANs: learning to generate images from new categories without forgetting
• Prototype-Guided Memory Replay for Continual Learning
• Subspace Regularizers for Few-Shot Class Incremental Learning
• MgSvF: Multi-Grained Slow vs. Fast Framework for Few-Shot Class-Incremental Learning
• Overcoming Catastrophic Forgetting in Incremental Few-Shot Learning by Finding Flat Minima
• GP-Tree: A Gaussian Process Classifier for Few-Shot Incremental Learning