EEC: Learning to Encode and Regenerate Images for Continual Learning

Abstract

The two main impediments to continual learning are catastrophic forgetting and memory limitations on the storage of data. To cope with these challenges, we propose a novel, cognitively-inspired approach which trains autoencoders with Neural Style Transfer to encode and store images. During training on a new task, reconstructed images from encoded episodes are replayed in order to avoid catastrophic forgetting. The loss function for the reconstructed images is weighted to reduce its effect during classifier training to cope with image degradation. When the system runs out of memory the encoded episodes are converted into centroids and covariance matrices, which are used to generate pseudo-images during classifier training, keeping classifier performance stable while using less memory. Our approach increases classification accuracy by 13-17% over state-of-the-art methods on benchmark datasets, while requiring 78% less storage space.

arXiv commentAdded link to the code in the paper. A preliminary version of this work was presented at ICML 2020 Workshop on Lifelong Machine Learning: arXiv:2007.06637

References in corpus (8)

• Distilling the Knowledge in a Neural Network
• Overcoming catastrophic forgetting in neural networks
• Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence
• FearNet: Brain-Inspired Model for Incremental Learning
• Continual Learning in Generative Adversarial Nets
• Incremental Classifier Learning with Generative Adversarial Networks
• Cognitively-Inspired Model for Incremental Learning Using a Few Examples
• Storing Encoded Episodes as Concepts for Continual Learning

Cited by in corpus (8)

• Class Gradient Projection For Continual Learning
• F-SIOL-310: A Robotic Dataset and Benchmark for Few-Shot Incremental Object Learning
• Continual Referring Expression Comprehension via Dual Modular Memorization
• CUCL: Codebook for Unsupervised Continual Learning
• CBCL-PR: A Cognitively Inspired Model for Class-Incremental Learning in Robotics
• RECALL: Rehearsal-free Continual Learning for Object Classification
• Learning Novel Objects Continually Through Curiosity
• IB-DRR: Incremental Learning with Information-Back Discrete Representation Replay