Graph-Based Continual Learning
arXiv:2007.04813
Abstract
Despite significant advances, continual learning models still suffer from catastrophic forgetting when exposed to incrementally available data from non-stationary distributions. Rehearsal approaches alleviate the problem by maintaining and replaying a small episodic memory of previous samples, often implemented as an array of independent memory slots. In this work, we propose to augment such an array with a learnable random graph that captures pairwise similarities between its samples, and use it not only to learn new tasks but also to guard against forgetting. Empirical results on several benchmark datasets show that our model consistently outperforms recently proposed baselines for task-free continual learning.
arXiv commentPublished as a conference paper at ICLR 2021
References in corpus (24)
- Distilling the Knowledge in a Neural Network
- Overcoming catastrophic forgetting in neural networks
- Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
- Categorical Reparameterization with Gumbel-Softmax
- Self-Attention Generative Adversarial Networks
- Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence
- The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables
- Continual Learning Through Synaptic Intelligence
- An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks
- PathNet: Evolution Channels Gradient Descent in Super Neural Networks
- Efficient Lifelong Learning with A-GEM
- On Tiny Episodic Memories in Continual Learning
- FearNet: Brain-Inspired Model for Incremental Learning
- Variational Continual Learning
- Re-evaluating Continual Learning Scenarios: A Categorization and Case for Strong Baselines
- Online Structured Laplace Approximations For Overcoming Catastrophic Forgetting
- Lifelong Learning with Dynamically Expandable Networks
- A Neural Dirichlet Process Mixture Model for Task-Free Continual Learning
- Continual Unsupervised Representation Learning
- Functional Regularisation for Continual Learning with Gaussian Processes
- Life-Long Disentangled Representation Learning with Cross-Domain Latent Homologies
- Online Learned Continual Compression with Adaptive Quantization Modules
- The Functional Neural Process
- Memory-based Parameter Adaptation