Alleviating catastrophic forgetting using context-dependent gating and synaptic stabilization

Abstract

Humans and most animals can learn new tasks without forgetting old ones. However, training artificial neural networks (ANNs) on new tasks typically cause it to forget previously learned tasks. This phenomenon is the result of "catastrophic forgetting", in which training an ANN disrupts connection weights that were important for solving previous tasks, degrading task performance. Several recent studies have proposed methods to stabilize connection weights of ANNs that are deemed most important for solving a task, which helps alleviate catastrophic forgetting. Here, drawing inspiration from algorithms that are believed to be implemented in vivo, we propose a complementary method: adding a context-dependent gating signal, such that only sparse, mostly non-overlapping patterns of units are active for any one task. This method is easy to implement, requires little computational overhead, and allows ANNs to maintain high performance across large numbers of sequentially presented tasks when combined with weight stabilization. This work provides another example of how neuroscience-inspired algorithms can benefit ANN design and capability.

arXiv commentPublished in PNAS, https://www.pnas.org/content/115/44/E10467

References in corpus (14)

• Adam: A Method for Stochastic Optimization
• TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
• Overcoming catastrophic forgetting in neural networks
• Dynamic reconfiguration of human brain networks during learning
• High-Dimensional Continuous Control Using Generalized Advantage Estimation
• Progressive Neural Networks
• Continual Learning Through Synaptic Intelligence
• An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks
• PathNet: Evolution Channels Gradient Descent in Super Neural Networks
• One-shot Learning with Memory-Augmented Neural Networks
• Variational Continual Learning
• Overcoming catastrophic forgetting with hard attention to the task
• Diffusion-based neuromodulation can eliminate catastrophic forgetting in simple neural networks
• Overcoming Catastrophic Interference by Conceptors

Cited by in corpus (52)

• Artificial neural networks for neuroscientists: A primer
• Continual Learning of Context-dependent Processing in Neural Networks
• Generative replay with feedback connections as a general strategy for continual learning
• Learning to Continually Learn
• Task Agnostic Continual Learning via Meta Learning
• Overcoming Long-term Catastrophic Forgetting through Adversarial Neural Pruning and Synaptic Consolidation
• Avoiding Catastrophe: Active Dendrites Enable Multi-Task Learning in Dynamic Environments
• Continual Learning of a Mixed Sequence of Similar and Dissimilar Tasks
• Connectivity Concepts in Neuronal Network Modeling
• Continual Learning and Catastrophic Forgetting
• Learning offline: memory replay in biological and artificial reinforcement learning
• Intrinsic Motivation and Episodic Memories for Robot Exploration of High-Dimensional Sensory Spaces
• Continual Learning via Inter-Task Synaptic Mapping
• Beneficial Perturbation Network for designing general adaptive artificial intelligence systems
• Knowledge Infused Learning (K-IL): Towards Deep Incorporation of Knowledge in Deep Learning
• Continual Learning in Recurrent Neural Networks
• Linear Mode Connectivity in Multitask and Continual Learning
• Advances in Continual Graph Learning for Anti-Money Laundering Systems: A Comprehensive Review
• Generalisation Guarantees for Continual Learning with Orthogonal Gradient Descent
• Modularity in Deep Learning: A Survey
• Statistical mechanics of continual learning: variational principle and mean-field potential
• Self-Supervised Learning Aided Class-Incremental Lifelong Learning
• If deep learning is the answer, then what is the question?
• Supermasks in Superposition
• Energy-Based Models for Continual Learning
• Robust neural circuit reconstruction from serial electron microscopy with convolutional recurrent networks
• OpenLORIS-Object: A Robotic Vision Dataset and Benchmark for Lifelong Deep Learning
• Continual Learning Using Multi-view Task Conditional Neural Networks
• Lifelong Learning using Eigentasks: Task Separation, Skill Acquisition, and Selective Transfer
• Overcoming Catastrophic Forgetting by Neuron-level Plasticity Control
• GAN Memory with No Forgetting
• Generative Memory for Lifelong Reinforcement Learning
• Optimizing Reusable Knowledge for Continual Learning via Metalearning
• IROS 2019 Lifelong Robotic Vision Challenge -- Lifelong Object Recognition Report
• DIRA: Dynamic Domain Incremental Regularised Adaptation
• Self-Net: Lifelong Learning via Continual Self-Modeling
• Monitoring nonstationary processes based on recursive cointegration analysis and elastic weight consolidation
• Generative Feature Replay with Orthogonal Weight Modification for Continual Learning
• A good body is all you need: avoiding catastrophic interference via agent architecture search
• Dendritic Self-Organizing Maps for Continual Learning
• Hybrid Backpropagation Parallel Reservoir Networks
• Better Knowledge Retention through Metric Learning
• Adversarial Incremental Learning
• Do Not Forget to Attend to Uncertainty while Mitigating Catastrophic Forgetting
• ZS-IL: Looking Back on Learned Experiences For Zero-Shot Incremental Learning
• Self-Constructing Neural Networks Through Random Mutation
• Embodiment dictates learnability in neural controllers
• Continual learning under domain transfer with sparse synaptic bursting
• Does the Adam Optimizer Exacerbate Catastrophic Forgetting?
• RSAC: Regularized Subspace Approximation Classifier for Lightweight Continuous Learning
• Investigating the generative dynamics of energy-based neural networks
• Context selectivity with dynamic availability enables lifelong continual learning