Incremental Meta-Learning via Indirect Discriminant Alignment

Abstract

Majority of the modern meta-learning methods for few-shot classification tasks operate in two phases: a meta-training phase where the meta-learner learns a generic representation by solving multiple few-shot tasks sampled from a large dataset and a testing phase, where the meta-learner leverages its learnt internal representation for a specific few-shot task involving classes which were not seen during the meta-training phase. To the best of our knowledge, all such meta-learning methods use a single base dataset for meta-training to sample tasks from and do not adapt the algorithm after meta-training. This strategy may not scale to real-world use-cases where the meta-learner does not potentially have access to the full meta-training dataset from the very beginning and we need to update the meta-learner in an incremental fashion when additional training data becomes available. Through our experimental setup, we develop a notion of incremental learning during the meta-training phase of meta-learning and propose a method which can be used with multiple existing metric-based meta-learning algorithms. Experimental results on benchmark dataset show that our approach performs favorably at test time as compared to training a model with the full meta-training set and incurs negligible amount of catastrophic forgetting

References in corpus (13)

• Distilling the Knowledge in a Neural Network
• Overcoming catastrophic forgetting in neural networks
• Prototypical Networks for Few-shot Learning
• How transferable are features in deep neural networks?
• Trust Region Policy Optimization
• Do Deep Nets Really Need to be Deep?
• Theoretical Models of Learning to Learn
• Efficient Lifelong Learning with A-GEM
• DropBlock: A regularization method for convolutional networks
• The Devil is in the Tails: Fine-grained Classification in the Wild
• Online Meta-Learning
• Adaptive Masked Proxies for Few-Shot Segmentation
• Continuous Meta-Learning without Tasks

Cited by in corpus (1)

• Few-shot Continual Learning: a Brain-inspired Approach