Formalizing the Generalization-Forgetting Trade-off in Continual Learning
arXiv:2109.14035
Abstract
We formulate the continual learning (CL) problem via dynamic programming and model the trade-off between catastrophic forgetting and generalization as a two-player sequential game. In this approach, player 1 maximizes the cost due to lack of generalization whereas player 2 minimizes the cost due to catastrophic forgetting. We show theoretically that a balance point between the two players exists for each task and that this point is stable (once the balance is achieved, the two players stay at the balance point). Next, we introduce balanced continual learning (BCL), which is designed to attain balance between generalization and forgetting and empirically demonstrate that BCL is comparable to or better than the state of the art.
References in corpus (10)
- On Tiny Episodic Memories in Continual Learning
- Online Meta-Learning
- Understanding the Role of Training Regimes in Continual Learning
- Learning to Continually Learn
- Continual Deep Learning by Functional Regularisation of Memorable Past
- Continual Learning of a Mixed Sequence of Similar and Dissimilar Tasks
- La-MAML: Look-ahead Meta Learning for Continual Learning
- Optimal Continual Learning has Perfect Memory and is NP-hard
- Optimization and Generalization of Regularization-Based Continual Learning: a Loss Approximation Viewpoint
- Generalisation Guarantees for Continual Learning with Orthogonal Gradient Descent