Optimization and Generalization of Regularization-Based Continual Learning: a Loss Approximation Viewpoint

Abstract

Neural networks have achieved remarkable success in many cognitive tasks. However, when they are trained sequentially on multiple tasks without access to old data, their performance on early tasks tend to drop significantly. This problem is often referred to as catastrophic forgetting, a key challenge in continual learning of neural networks. The regularization-based approach is one of the primary classes of methods to alleviate catastrophic forgetting. In this paper, we provide a novel viewpoint of regularization-based continual learning by formulating it as a second-order Taylor approximation of the loss function of each task. This viewpoint leads to a unified framework that can be instantiated to derive many existing algorithms such as Elastic Weight Consolidation and Kronecker factored Laplace approximation. Based on this viewpoint, we study the optimization aspects (i.e., convergence) as well as generalization properties (i.e., finite-sample guarantees) of regularization-based continual learning. Our theoretical results indicate the importance of accurate approximation of the Hessian matrix. The experimental results on several benchmarks provide empirical validation of our theoretical findings.

arXiv commentPreliminary version with a different title presented at ICML Workshop on Continual Learning, 2020 (spotlight)

References in corpus (15)

• Overcoming catastrophic forgetting in neural networks
• Neural Tangent Kernel: Convergence and Generalization in Neural Networks
• Continual Learning Through Synaptic Intelligence
• Exact solutions to the nonlinear dynamics of learning in deep linear neural networks
• The Loss Surfaces of Multilayer Networks
• Optimizing Neural Networks with Kronecker-factored Approximate Curvature
• Variational Continual Learning
• Online Structured Laplace Approximations For Overcoming Catastrophic Forgetting
• Deep Generative Dual Memory Network for Continual Learning
• The Benefit of Multitask Representation Learning
• Understanding the Role of Training Regimes in Continual Learning
• Functional Regularisation for Continual Learning with Gaussian Processes
• Learn to Grow: A Continual Structure Learning Framework for Overcoming Catastrophic Forgetting
• A Unifying Bayesian View of Continual Learning
• Toward Understanding Catastrophic Forgetting in Continual Learning

Cited by in corpus (3)

• Formalizing the Generalization-Forgetting Trade-off in Continual Learning
• How do Quadratic Regularizers Prevent Catastrophic Forgetting: The Role of Interpolation
• Provable Lifelong Learning of Representations