Learning from the Past: Continual Meta-Learning via Bayesian Graph Modeling

Abstract

Meta-learning for few-shot learning allows a machine to leverage previously acquired knowledge as a prior, thus improving the performance on novel tasks with only small amounts of data. However, most mainstream models suffer from catastrophic forgetting and insufficient robustness issues, thereby failing to fully retain or exploit long-term knowledge while being prone to cause severe error accumulation. In this paper, we propose a novel Continual Meta-Learning approach with Bayesian Graph Neural Networks (CML-BGNN) that mathematically formulates meta-learning as continual learning of a sequence of tasks. With each task forming as a graph, the intra- and inter-task correlations can be well preserved via message-passing and history transition. To remedy topological uncertainty from graph initialization, we utilize Bayes by Backprop strategy that approximates the posterior distribution of task-specific parameters with amortized inference networks, which are seamlessly integrated into the end-to-end edge learning. Extensive experiments conducted on the miniImageNet and tieredImageNet datasets demonstrate the effectiveness and efficiency of the proposed method, improving the performance by 42.8% compared with state-of-the-art on the miniImageNet 5-way 1-shot classification task.

References in corpus (12)

• Prototypical Networks for Few-shot Learning
• A Simple Neural Attentive Meta-Learner
• Meta-SGD: Learning to Learn Quickly for Few-Shot Learning
• Learning to Propagate Labels: Transductive Propagation Network for Few-shot Learning
• Few-Shot Learning with Graph Neural Networks
• Meta-Learning with Latent Embedding Optimization
• Continual Learning with Deep Generative Replay
• Meta-learning with differentiable closed-form solvers
• Born to Learn: the Inspiration, Progress, and Future of Evolved Plastic Artificial Neural Networks
• Finding Task-Relevant Features for Few-Shot Learning by Category Traversal
• Low-Shot Learning from Imaginary Data
• Edge-labeling Graph Neural Network for Few-shot Learning

Cited by in corpus (4)

• A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges
• Online Fast Adaptation and Knowledge Accumulation: a New Approach to Continual Learning
• High-order structure preserving graph neural network for few-shot learning
• Entropy-Based Uncertainty Calibration for Generalized Zero-Shot Learning