Optimal Protocols for Continual Learning via Statistical Physics and Control Theory

Abstract

Artificial neural networks often struggle with catastrophic forgetting when learning multiple tasks sequentially, as training on new tasks degrades the performance on previously learned tasks. Recent theoretical work has addressed this issue by analysing learning curves in synthetic frameworks under predefined training protocols. However, these protocols relied on heuristics and lacked a solid theoretical foundation assessing their optimality. In this paper, we fill this gap by combining exact equations for training dynamics, derived using statistical physics techniques, with optimal control methods. We apply this approach to teacher-student models for continual learning and multi-task problems, obtaining a theory for task-selection protocols maximising performance while minimising forgetting. Our theoretical analysis offers non-trivial yet interpretable strategies for mitigating catastrophic forgetting, shedding light on how optimal learning protocols modulate established effects, such as the influence of task similarity on forgetting. Finally, we validate our theoretical findings with experiments on real-world data.

22 pages, 12 figures

References in corpus (32)

• Overcoming catastrophic forgetting in neural networks
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• A continual learning survey: Defying forgetting in classification tasks
• Progressive Neural Networks
• A Model of Inductive Bias Learning
• A Mean Field View of the Landscape of Two-Layers Neural Networks
• Experience Replay for Continual Learning
• Towards Robust Evaluations of Continual Learning
• Modelling the influence of data structure on learning in neural networks: the hidden manifold model
• On the Global Convergence of Gradient Descent for Over-parameterized Models using Optimal Transport
• An Investigation of Why Overparameterization Exacerbates Spurious Correlations
• A Mean-Field Optimal Control Formulation of Deep Learning
• Learning curves of generic features maps for realistic datasets with a teacher-student model
• Trainability and Accuracy of Neural Networks: An Interacting Particle System Approach
• Mapping of attention mechanisms to a generalized Potts model
• Probing transfer learning with a model of synthetic correlated datasets
• On The Impact of Machine Learning Randomness on Group Fairness
• Self-Consistent Dynamical Field Theory of Kernel Evolution in Wide Neural Networks
• Phase Transitions in Transfer Learning for High-Dimensional Perceptrons
• Phase diagram of Stochastic Gradient Descent in high-dimensional two-layer neural networks
• An Analytical Theory of Curriculum Learning in Teacher-Student Networks
• The Influence of Learning Rule on Representation Dynamics in Wide Neural Networks
• Forward Learning with Top-Down Feedback: Empirical and Analytical Characterization
• Optimisation of on-line principal component analysis
• Disordered high-dimensional optimal control
• Maslow's Hammer for Catastrophic Forgetting: Node Re-Use vs Node Activation
• Provable Advantage of Curriculum Learning on Parity Targets with Mixed Inputs
• Dissecting the Interplay of Attention Paths in a Statistical Mechanics Theory of Transformers
• Why Do Animals Need Shaping? A Theory of Task Composition and Curriculum Learning
• Meta-Learning Strategies through Value Maximization in Neural Networks
• Bias in Motion: Theoretical Insights into the Dynamics of Bias in SGD Training
• Order parameters and phase transitions of continual learning in deep neural networks