Shared and Private VAEs with Generative Replay for Continual Learning

Abstract

Continual learning tries to learn new tasks without forgetting previously learned ones. In reality, most of the existing artificial neural network(ANN) models fail, while humans do the same by remembering previous works throughout their life. Although simply storing all past data can alleviate the problem, it needs large memory and often infeasible in real-world applications where last data access is limited. We hypothesize that the model that learns to solve each task continually has some task-specific properties and some task-invariant characteristics. We propose a hybrid continual learning model that is more suitable in real case scenarios to address the issues that has a task-invariant shared variational autoencoder and T task-specific variational autoencoders. Our model combines generative replay and architectural growth to prevent catastrophic forgetting. We show our hybrid model effectively avoids forgetting and achieves state-of-the-art results on visual continual learning benchmarks such as MNIST, Permuted MNIST(QMNIST), CIFAR100, and miniImageNet datasets. We discuss results on a few more datasets, such as SVHN, Fashion-MNIST, EMNIST, and CIFAR10.

arXiv comment10 pages, 10 figures

References in corpus (16)

• Adam: A Method for Stochastic Optimization
• Overcoming catastrophic forgetting in neural networks
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence
• Continual Learning Through Synaptic Intelligence
• A Survey on Multi-view Learning
• PathNet: Evolution Channels Gradient Descent in Super Neural Networks
• Gradient Episodic Memory for Continual Learning
• On Tiny Episodic Memories in Continual Learning
• FearNet: Brain-Inspired Model for Incremental Learning
• Variational Continual Learning
• Continual Learning with Deep Generative Replay
• Lifelong Learning with Dynamically Expandable Networks
• Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interference
• Memory Replay GANs: learning to generate images from new categories without forgetting
• Variational Prototype Replays for Continual Learning