Gradient based sample selection for online continual learning

Abstract

A continual learning agent learns online with a non-stationary and never-ending stream of data. The key to such learning process is to overcome the catastrophic forgetting of previously seen data, which is a well known problem of neural networks. To prevent forgetting, a replay buffer is usually employed to store the previous data for the purpose of rehearsal. Previous works often depend on task boundary and i.i.d. assumptions to properly select samples for the replay buffer. In this work, we formulate sample selection as a constraint reduction problem based on the constrained optimization view of continual learning. The goal is to select a fixed subset of constraints that best approximate the feasible region defined by the original constraints. We show that it is equivalent to maximizing the diversity of samples in the replay buffer with parameters gradient as the feature. We further develop a greedy alternative that is cheap and efficient. The advantage of the proposed method is demonstrated by comparing to other alternatives under the continual learning setting. Further comparisons are made against state of the art methods that rely on task boundaries which show comparable or even better results for our method.

arXiv commentNeurips 2019

References in corpus (11)

• Playing Atari with Deep Reinforcement Learning
• Continual Learning Through Synaptic Intelligence
• Efficient Lifelong Learning with A-GEM
• On Tiny Episodic Memories in Continual Learning
• Variational Continual Learning
• BAGAN: Data Augmentation with Balancing GAN
• Continual Learning with Deep Generative Replay
• Reinforced Continual Learning
• Towards Robust Evaluations of Continual Learning
• Selfless Sequential Learning
• Positivity theorems for solid-angle polynomials

Cited by in corpus (12)

• A continual learning survey: Defying forgetting in classification tasks
• Understanding the Role of Training Regimes in Continual Learning
• Online Continual Learning with Maximally Interfered Retrieval
• Recent Advances of Continual Learning in Computer Vision: An Overview
• Continual Learning in Low-rank Orthogonal Subspaces
• Complementary Calibration: Boosting General Continual Learning with Collaborative Distillation and Self-Supervision
• CoDEPS: Online Continual Learning for Depth Estimation and Panoptic Segmentation
• Linear Mode Connectivity in Multitask and Continual Learning
• Learning to Recommend via Meta Parameter Partition
• Single-Net Continual Learning with Progressive Segmented Training (PST)
• Subjectivity Learning Theory towards Artificial General Intelligence
• ZS-IL: Looking Back on Learned Experiences For Zero-Shot Incremental Learning