Learning to Continually Learn

Abstract

Continual lifelong learning requires an agent or model to learn many sequentially ordered tasks, building on previous knowledge without catastrophically forgetting it. Much work has gone towards preventing the default tendency of machine learning models to catastrophically forget, yet virtually all such work involves manually-designed solutions to the problem. We instead advocate meta-learning a solution to catastrophic forgetting, allowing AI to learn to continually learn. Inspired by neuromodulatory processes in the brain, we propose A Neuromodulated Meta-Learning Algorithm (ANML). It differentiates through a sequential learning process to meta-learn an activation-gating function that enables context-dependent selective activation within a deep neural network. Specifically, a neuromodulatory (NM) neural network gates the forward pass of another (otherwise normal) neural network called the prediction learning network (PLN). The NM network also thus indirectly controls selective plasticity (i.e. the backward pass of) the PLN. ANML enables continual learning without catastrophic forgetting at scale: it produces state-of-the-art continual learning performance, sequentially learning as many as 600 classes (over 9,000 SGD updates).

References in corpus (16)

• Adam: A Method for Stochastic Optimization
• Overcoming catastrophic forgetting in neural networks
• Neural Architecture Search with Reinforcement Learning
• How transferable are features in deep neural networks?
• Prioritized Experience Replay
• Continual Learning Through Synaptic Intelligence
• PathNet: Evolution Channels Gradient Descent in Super Neural Networks
• Efficient Neural Architecture Search via Parameter Sharing
• RL${}^2$: Fast Reinforcement Learning via Slow Reinforcement Learning
• Learning to reinforcement learn
• Alleviating catastrophic forgetting using context-dependent gating and synaptic stabilization
• Conditional Computation in Neural Networks for faster models
• Born to Learn: the Inspiration, Progress, and Future of Evolved Plastic Artificial Neural Networks
• AI-GAs: AI-generating algorithms, an alternate paradigm for producing general artificial intelligence
• Diffusion-based neuromodulation can eliminate catastrophic forgetting in simple neural networks
• Backpropamine: training self-modifying neural networks with differentiable neuromodulated plasticity

Cited by in corpus (32)

• Continual Learning for Recurrent Neural Networks: an Empirical Evaluation
• Understanding the Role of Training Regimes in Continual Learning
• AI-GAs: AI-generating algorithms, an alternate paradigm for producing general artificial intelligence
• Recent Advances of Continual Learning in Computer Vision: An Overview
• Continual Learning in Low-rank Orthogonal Subspaces
• Meta-Consolidation for Continual Learning
• Meta-Learning with Sparse Experience Replay for Lifelong Language Learning
• Physics-integrated machine learning: embedding a neural network in the Navier-Stokes equations. Part II
• Physics-integrated machine learning: embedding a neural network in the Navier-Stokes equations. Part I
• Online Fast Adaptation and Knowledge Accumulation: a New Approach to Continual Learning
• Linear Mode Connectivity in Multitask and Continual Learning
• A Policy Gradient Algorithm for Learning to Learn in Multiagent Reinforcement Learning
• Context Meta-Reinforcement Learning via Neuromodulation
• Learning Online Visual Invariances for Novel Objects via Supervised and Self-Supervised Training
• ARCADe: A Rapid Continual Anomaly Detector
• Memory-Efficient Semi-Supervised Continual Learning: The World is its Own Replay Buffer
• Continual Learning with Deep Artificial Neurons
• Formalizing the Generalization-Forgetting Trade-off in Continual Learning
• DRILL: Dynamic Representations for Imbalanced Lifelong Learning
• Wide Neural Networks Forget Less Catastrophically
• Meta Continual Learning via Dynamic Programming
• MPLP: Learning a Message Passing Learning Protocol
• Neuromodulated Neural Architectures with Local Error Signals for Memory-Constrained Online Continual Learning
• Continual Learning: Tackling Catastrophic Forgetting in Deep Neural Networks with Replay Processes
• Learning Translation Invariance in CNNs
• Few-shot Continual Learning: a Brain-inspired Approach
• Federated Reconnaissance: Efficient, Distributed, Class-Incremental Learning
• Bilevel Continual Learning
• Learning to Learn to be Right for the Right Reasons
• Condensed Composite Memory Continual Learning
• Visual Goal-Directed Meta-Learning with Contextual Planning Networks
• A Meta-Learned Neuron model for Continual Learning