Flattening Sharpness for Dynamic Gradient Projection Memory Benefits Continual Learning
arXiv:2110.04593
Abstract
The backpropagation networks are notably susceptible to catastrophic forgetting, where networks tend to forget previously learned skills upon learning new ones. To address such the 'sensitivity-stability' dilemma, most previous efforts have been contributed to minimizing the empirical risk with different parameter regularization terms and episodic memory, but rarely exploring the usages of the weight loss landscape. In this paper, we investigate the relationship between the weight loss landscape and sensitivity-stability in the continual learning scenario, based on which, we propose a novel method, Flattening Sharpness for Dynamic Gradient Projection Memory (FS-DGPM). In particular, we introduce a soft weight to represent the importance of each basis representing past tasks in GPM, which can be adaptively learned during the learning process, so that less important bases can be dynamically released to improve the sensitivity of new skill learning. We further introduce Flattening Sharpness (FS) to reduce the generalization gap by explicitly regulating the flatness of the weight loss landscape of all seen tasks. As demonstrated empirically, our proposed method consistently outperforms baselines with the superior ability to learn new skills while alleviating forgetting effectively.
NeurIPS2021
References in corpus (15)
- Overcoming catastrophic forgetting in neural networks
- Continual Learning Through Synaptic Intelligence
- Gradient Episodic Memory for Continual Learning
- Variational Continual Learning
- Continual Learning with Deep Generative Replay
- Reinforced Continual Learning
- Flipout: Efficient Pseudo-Independent Weight Perturbations on Mini-Batches
- Lifelong Learning with Dynamically Expandable Networks
- Sharpness-Aware Minimization for Efficiently Improving Generalization
- Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interference
- Understanding the Role of Training Regimes in Continual Learning
- La-MAML: Look-ahead Meta Learning for Continual Learning
- CPR: Classifier-Projection Regularization for Continual Learning
- Scalable and Order-robust Continual Learning with Additive Parameter Decomposition
- Graph-Based Continual Learning