Localizing Catastrophic Forgetting in Neural Networks

Abstract

Artificial neural networks (ANNs) suffer from catastrophic forgetting when trained on a sequence of tasks. While this phenomenon was studied in the past, there is only very limited recent research on this phenomenon. We propose a method for determining the contribution of individual parameters in an ANN to catastrophic forgetting. The method is used to analyze an ANNs response to three different continual learning scenarios.

References in corpus (6)

• Overcoming catastrophic forgetting in neural networks
• Continual Learning Through Synaptic Intelligence
• An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks
• Re-evaluating Continual Learning Scenarios: A Categorization and Case for Strong Baselines
• Generative replay with feedback connections as a general strategy for continual learning
• Towards Robust Evaluations of Continual Learning

Cited by in corpus (3)

• Elastic Weight Consolidation (EWC): Nuts and Bolts
• Privacy-preserving Continual Federated Clustering via Adaptive Resonance Theory
• Association: Remind Your GAN not to Forget