Continual Learning Through Synaptic Intelligence

Abstract

While deep learning has led to remarkable advances across diverse applications, it struggles in domains where the data distribution changes over the course of learning. In stark contrast, biological neural networks continually adapt to changing domains, possibly by leveraging complex molecular machinery to solve many tasks simultaneously. In this study, we introduce intelligent synapses that bring some of this biological complexity into artificial neural networks. Each synapse accumulates task relevant information over time, and exploits this information to rapidly store new memories without forgetting old ones. We evaluate our approach on continual learning of classification tasks, and show that it dramatically reduces forgetting while maintaining computational efficiency.

ICML 2017

Cited by in corpus (356)

• A continual learning survey: Defying forgetting in classification tasks
• Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence
• Three scenarios for continual learning
• Artificial neural networks for neuroscientists: A primer
• Experience Replay for Continual Learning
• On Tiny Episodic Memories in Continual Learning
• Alleviating catastrophic forgetting using context-dependent gating and synaptic stabilization
• Re-evaluating Continual Learning Scenarios: A Categorization and Case for Strong Baselines
• Reinforced Continual Learning
• Generative replay with feedback connections as a general strategy for continual learning
• Towards Robust Evaluations of Continual Learning
• Incremental Learning for Semantic Segmentation of Large-Scale Remote Sensing Data
• Parameter-Efficient Transfer Learning for NLP
• Compacting, Picking and Growing for Unforgetting Continual Learning
• Deep Generative Dual Memory Network for Continual Learning
• Continual Learning for Recurrent Neural Networks: an Empirical Evaluation
• Variational Federated Multi-Task Learning
• LAMOL: LAnguage MOdeling for Lifelong Language Learning
• Lifelong Generative Modeling
• Incremental Object Detection via Meta-Learning
• Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interference
• Implicit Generation and Generalization in Energy-Based Models
• Don't forget, there is more than forgetting: new metrics for Continual Learning
• Lifelong Learning of Spatiotemporal Representations with Dual-Memory Recurrent Self-Organization
• A Neural Dirichlet Process Mixture Model for Task-Free Continual Learning
• Dark Experience for General Continual Learning: a Strong, Simple Baseline
• Functional Regularisation for Continual Learning with Gaussian Processes
• Coresets via Bilevel Optimization for Continual Learning and Streaming
• Synaptic metaplasticity in binarized neural networks
• Gradient based sample selection for online continual learning
• LEEP: A New Measure to Evaluate Transferability of Learned Representations
• Task Agnostic Continual Learning Using Online Variational Bayes
• Learning to Continually Learn
• SpaceNet: Make Free Space For Continual Learning
• Lifelong Teacher-Student Network Learning
• DualNet: Continual Learning, Fast and Slow
• Domain-Incremental Continual Learning for Mitigating Bias in Facial Expression and Action Unit Recognition
• Task Agnostic Continual Learning via Meta Learning
• Overcoming Long-term Catastrophic Forgetting through Adversarial Neural Pruning and Synaptic Consolidation
• Avoiding Catastrophe: Active Dendrites Enable Multi-Task Learning in Dynamic Environments
• Continual Learning of a Mixed Sequence of Similar and Dissimilar Tasks
• What shapes feature representations? Exploring datasets, architectures, and training
• AI-GAs: AI-generating algorithms, an alternate paradigm for producing general artificial intelligence
• A Unifying Bayesian View of Continual Learning
• Efficient Continual Learning in Neural Networks with Embedding Regularization
• Deep Online Learning via Meta-Learning: Continual Adaptation for Model-Based RL
• Dreaming to Distill: Data-free Knowledge Transfer via DeepInversion
• Learning to Continuously Optimize Wireless Resource in a Dynamic Environment: A Bilevel Optimization Perspective
• Lifelong Machine Learning Potentials
• Inexact-ADMM Based Federated Meta-Learning for Fast and Continual Edge Learning
• CLeaR: An Adaptive Continual Learning Framework for Regression Tasks
• DER: Dynamically Expandable Representation for Class Incremental Learning
• Recent Advances of Continual Learning in Computer Vision: An Overview
• Orthogonal Gradient Descent for Continual Learning
• Learning offline: memory replay in biological and artificial reinforcement learning
• Unified Probabilistic Deep Continual Learning through Generative Replay and Open Set Recognition
• Online Continual Learning on Sequences
• Rotate your Networks: Better Weight Consolidation and Less Catastrophic Forgetting
• DisCoRL: Continual Reinforcement Learning via Policy Distillation
• Multi-Task Incremental Learning for Object Detection
• Pretraining Representations for Data-Efficient Reinforcement Learning
• A distillation-based approach integrating continual learning and federated learning for pervasive services
• Class-incremental Learning with Pre-allocated Fixed Classifiers
• An Investigation of Replay-based Approaches for Continual Learning
• Toward Understanding Catastrophic Forgetting in Continual Learning
• CPR: Classifier-Projection Regularization for Continual Learning
• Maintaining Discrimination and Fairness in Class Incremental Learning
• Federated Learning with Additional Mechanisms on Clients to Reduce Communication Costs
• Continual Learning in Low-rank Orthogonal Subspaces
• Improving and Understanding Variational Continual Learning
• Continual Learning with Gated Incremental Memories for sequential data processing
• Continuous Domain Adaptation with Variational Domain-Agnostic Feature Replay
• Efficient Continual Learning with Modular Networks and Task-Driven Priors
• Class-incremental Learning via Deep Model Consolidation
• Meta-Consolidation for Continual Learning
• Continual Learning with Adaptive Weights (CLAW)
• Improved Schemes for Episodic Memory-based Lifelong Learning
• Continual Learning with Node-Importance based Adaptive Group Sparse Regularization
• RODEO: Replay for Online Object Detection
• Defining Benchmarks for Continual Few-Shot Learning
• Gradient Projection Memory for Continual Learning
• Toward Continual Learning for Conversational Agents
• Deep Reinforcement Learning amidst Lifelong Non-Stationarity
• Uncertainty-guided Continual Learning with Bayesian Neural Networks
• Meta Continual Learning
• RATT: Recurrent Attention to Transient Tasks for Continual Image Captioning
• Meta-Learning Representations for Continual Learning
• Continual Learning via Inter-Task Synaptic Mapping
• Online Optimization with Predictions and Switching Costs: Fast Algorithms and the Fundamental Limit
• Lifelong Object Detection
• Flattening Sharpness for Dynamic Gradient Projection Memory Benefits Continual Learning
• On Catastrophic Forgetting and Mode Collapse in Generative Adversarial Networks
• Learning to Continuously Optimize Wireless Resource In Episodically Dynamic Environment
• A Review of Single-Source Deep Unsupervised Visual Domain Adaptation
• Self-Supervised Training Enhances Online Continual Learning
• Anatomy of Catastrophic Forgetting: Hidden Representations and Task Semantics
• Continual Reinforcement Learning with Complex Synapses
• Replay in Deep Learning: Current Approaches and Missing Biological Elements
• REMIND Your Neural Network to Prevent Catastrophic Forgetting
• Elastic Weight Consolidation (EWC): Nuts and Bolts
• AirLoop: Lifelong Loop Closure Detection
• Regularization Shortcomings for Continual Learning
• Continuous learning of spiking networks trained with local rules
• The Effectiveness of Memory Replay in Large Scale Continual Learning
• Rainbow Memory: Continual Learning with a Memory of Diverse Samples
• Progressive Memory Banks for Incremental Domain Adaptation
• Contrastive Syn-to-Real Generalization
• Continual Classification Learning Using Generative Models
• Measuring and regularizing networks in function space
• An Adaptive Random Path Selection Approach for Incremental Learning
• Two-Level Residual Distillation based Triple Network for Incremental Object Detection
• Training Binary Neural Networks using the Bayesian Learning Rule
• Task Agnostic Continual Learning Using Online Variational Bayes with Fixed-Point Updates
• Generalized Variational Continual Learning
• Batch-level Experience Replay with Review for Continual Learning
• Carousel Memory: Rethinking the Design of Episodic Memory for Continual Learning
• Few-Shot Class-Incremental Learning
• One Person, One Model, One World: Learning Continual User Representation without Forgetting
• Meta-Learning with Sparse Experience Replay for Lifelong Language Learning
• A Theoretical Analysis of Catastrophic Forgetting through the NTK Overlap Matrix
• Semantic Drift Compensation for Class-Incremental Learning
• Online Class-Incremental Continual Learning with Adversarial Shapley Value
• Online Continual Learning via the Knowledge Invariant and Spread-out Properties
• Optimization and Generalization of Regularization-Based Continual Learning: a Loss Approximation Viewpoint
• Deep learning via message passing algorithms based on belief propagation
• ACE: Adapting to Changing Environments for Semantic Segmentation
• Few-Shot Self Reminder to Overcome Catastrophic Forgetting
• Continual Learning in Recurrent Neural Networks
• Reconciling meta-learning and continual learning with online mixtures of tasks
• Sentence Embedding Alignment for Lifelong Relation Extraction
• ConDA: Continual Unsupervised Domain Adaptation
• Linear Mode Connectivity in Multitask and Continual Learning
• Estimating Model Uncertainty of Neural Networks in Sparse Information Form
• Lifelong Machine Learning with Deep Streaming Linear Discriminant Analysis
• Input-Driven Dynamics for Robust Memory Retrieval in Hopfield Networks
• Lifelong Policy Gradient Learning of Factored Policies for Faster Training Without Forgetting
• Leveraging Old Knowledge to Continually Learn New Classes in Medical Images
• Tackling Catastrophic Forgetting and Background Shift in Continual Semantic Segmentation
• Generalisation Guarantees for Continual Learning with Orthogonal Gradient Descent
• Lifelong Learning of Compositional Structures
• Variational Prototype Replays for Continual Learning
• Compositional Generalization for Primitive Substitutions
• ContCap: A scalable framework for continual image captioning
• Incremental Learning with Maximum Entropy Regularization: Rethinking Forgetting and Intransigence
• On the role of neurogenesis in overcoming catastrophic forgetting
• Hierarchical Indian Buffet Neural Networks for Bayesian Continual Learning
• Training Networks in Null Space of Feature Covariance for Continual Learning
• Eight challenges in developing theory of intelligence
• Generative Feature Replay For Class-Incremental Learning
• Supervised Contrastive Replay: Revisiting the Nearest Class Mean Classifier in Online Class-Incremental Continual Learning
• Recall and Learn: Fine-tuning Deep Pretrained Language Models with Less Forgetting
• Encoders and Ensembles for Task-Free Continual Learning
• Federated Intrusion Detection for IoT with Heterogeneous Cohort Privacy
• Lifelong Graph Learning
• Gradient Episodic Memory with a Soft Constraint for Continual Learning
• Continual Reinforcement Learning with Multi-Timescale Replay
• Half-Real Half-Fake Distillation for Class-Incremental Semantic Segmentation
• Gradient Regularized Contrastive Learning for Continual Domain Adaptation
• Online Structured Meta-learning
• ARCADe: A Rapid Continual Anomaly Detector
• Hypernetworks for Continual Semi-Supervised Learning
• Continual Domain-Tuning for Pretrained Language Models
• Continual Learning using a Bayesian Nonparametric Dictionary of Weight Factors
• Self-Supervised Learning Aided Class-Incremental Lifelong Learning
• Supermasks in Superposition
• Modular Meta-Learning with Shrinkage
• Autoencoder-Based Incremental Class Learning without Retraining on Old Data
• An Analytical Theory of Curriculum Learning in Teacher-Student Networks
• A Lifelong Learning Approach to Mobile Robot Navigation
• Memory Efficient Experience Replay for Streaming Learning
• Dissecting Catastrophic Forgetting in Continual Learning by Deep Visualization
• Meta-Learning-Based Robust Adaptive Flight Control Under Uncertain Wind Conditions
• Memory-Efficient Semi-Supervised Continual Learning: The World is its Own Replay Buffer
• Continual Learning for Sentence Representations Using Conceptors
• On Sequential Bayesian Inference for Continual Learning
• Evaluating Online Continual Learning with CALM
• Continuous Coordination As a Realistic Scenario for Lifelong Learning
• Artificial Neural Variability for Deep Learning: On Overfitting, Noise Memorization, and Catastrophic Forgetting
• L3DOC: Lifelong 3D Object Classification
• Self-Supervised GANs via Auxiliary Rotation Loss
• Lifelong Bayesian Optimization
• Automatic Recall Machines: Internal Replay, Continual Learning and the Brain
• Understanding Continual Learning Settings with Data Distribution Drift Analysis
• Continual Model-Based Reinforcement Learning with Hypernetworks
• Lifelong Neural Predictive Coding: Learning Cumulatively Online without Forgetting
• Energy-Based Models for Continual Learning
• Differentially Private Continual Learning
• ModelCI-e: Enabling Continual Learning in Deep Learning Serving Systems
• Lifelong Learning of Hate Speech Classification on Social Media
• Continual Representation Learning for Biometric Identification
• Bayesian Optimized Continual Learning with Attention Mechanism
• Continual Learning in Deep Neural Network by Using a Kalman Optimiser
• Towards Fair Affective Robotics: Continual Learning for Mitigating Bias in Facial Expression and Action Unit Recognition
• Continual Domain Adaptation for Machine Reading Comprehension
• BI-MAML: Balanced Incremental Approach for Meta Learning
• CUCL: Codebook for Unsupervised Continual Learning
• Distributed Weight Consolidation: A Brain Segmentation Case Study
• Continual egocentric object recognition
• Positive-Congruent Training: Towards Regression-Free Model Updates
• Localizing Catastrophic Forgetting in Neural Networks
• Continual Learning via Online Leverage Score Sampling
• Unifying Regularisation Methods for Continual Learning
• GAN Cocktail: mixing GANs without dataset access
• Multi-Domain Multi-Task Rehearsal for Lifelong Learning
• Pseudo-Rehearsal for Continual Learning with Normalizing Flows
• Online Continual Learning on Class Incremental Blurry Task Configuration with Anytime Inference
• Update Frequently, Update Fast: Retraining Semantic Parsing Systems in a Fraction of Time
• Continual Learning Using Bayesian Neural Networks
• OpenLORIS-Object: A Robotic Vision Dataset and Benchmark for Lifelong Deep Learning
• DRILL: Dynamic Representations for Imbalanced Lifelong Learning
• Thalamocortical contribution to solving credit assignment in neural systems
• Learnable Expansion-and-Compression Network for Few-shot Class-Incremental Learning
• Learning Invariant Representation for Continual Learning
• Lifelong Learning with Sketched Structural Regularization
• Continual Learning with Fully Probabilistic Models
• Online Continual Learning with Natural Distribution Shifts: An Empirical Study with Visual Data
• Layerwise Optimization by Gradient Decomposition for Continual Learning
• Online Mutual Adaptation of Deep Depth Prediction and Visual SLAM
• Wide Neural Networks Forget Less Catastrophically
• Towards Better Plasticity-Stability Trade-off in Incremental Learning: A Simple Linear Connector
• Sequoia: A Software Framework to Unify Continual Learning Research
• Continual Learning for Text Classification with Information Disentanglement Based Regularization
• Posterior Meta-Replay for Continual Learning
• Continual Learning Using Multi-view Task Conditional Neural Networks
• A Combinatorial Perspective on Transfer Learning
• Meta Continual Learning via Dynamic Programming
• Few-Shot Unsupervised Continual Learning through Meta-Examples
• Visually Grounded Continual Learning of Compositional Phrases
• Lifelong Learning using Eigentasks: Task Separation, Skill Acquisition, and Selective Transfer
• Generalized Energy Based Models
• Overcoming Catastrophic Forgetting by Generative Regularization
• Continual Learning for Natural Language Generation in Task-oriented Dialog Systems
• Bayesian Structure Adaptation for Continual Learning
• Overcoming Catastrophic Forgetting by Neuron-level Plasticity Control
• Attention-Based Structural-Plasticity
• Continual Learning with Self-Organizing Maps
• One step back, two steps forward: interference and learning in recurrent neural networks
• A contrastive rule for meta-learning
• Neurocoder: Learning General-Purpose Computation Using Stored Neural Programs
• Label Mapping Neural Networks with Response Consolidation for Class Incremental Learning
• Efficient Meta Lifelong-Learning with Limited Memory
• Optimizing Reusable Knowledge for Continual Learning via Metalearning
• Hyperparameter-free Continuous Learning for Domain Classification in Natural Language Understanding
• Neuromodulated Neural Architectures with Local Error Signals for Memory-Constrained Online Continual Learning
• Rethinking Experience Replay: a Bag of Tricks for Continual Learning
• Continual Learning: Tackling Catastrophic Forgetting in Deep Neural Networks with Replay Processes
• GROWN: GRow Only When Necessary for Continual Learning
• Overcoming Catastrophic Forgetting by Soft Parameter Pruning
• Single-Net Continual Learning with Progressive Segmented Training (PST)
• GAN Memory with No Forgetting
• Frosting Weights for Better Continual Training
• Facilitating Bayesian Continual Learning by Natural Gradients and Stein Gradients
• Learn-Prune-Share for Lifelong Learning
• Reviewing continual learning from the perspective of human-level intelligence
• No Free Lunch: Balancing Learning and Exploitation at the Network Edge
• ADER: Adaptively Distilled Exemplar Replay Towards Continual Learning for Session-based Recommendation
• Co-Transport for Class-Incremental Learning
• A Deep Learning Framework for Lifelong Machine Learning
• Memory Efficient Class-Incremental Learning for Image Classification
• Continual Learning with Extended Kronecker-factored Approximate Curvature
• On Catastrophic Interference in Atari 2600 Games
• Detecting and Adapting to Irregular Distribution Shifts in Bayesian Online Learning
• Learning to Remember from a Multi-Task Teacher
• Few-shot Continual Learning: a Brain-inspired Approach
• Learning Continually from Low-shot Data Stream
• Lifelong Learning with Searchable Extension Units
• Causal Attention for Unbiased Visual Recognition
• Representation Consolidation for Training Expert Students
• Class-wise Classifier Design Capable of Continual Learning using Adaptive Resonance Theory-based Topological Clustering
• Accretionary Learning with Deep Neural Networks
• A Conceptual Framework for Lifelong Learning
• Disentanglement of Color and Shape Representations for Continual Learning
• Federated Reconnaissance: Efficient, Distributed, Class-Incremental Learning
• Local learning rules to attenuate forgetting in neural networks
• Graph-Based Continual Learning
• Algorithmic insights on continual learning from fruit flies
• Neural Stored-program Memory
• Power Law in Sparsified Deep Neural Networks
• Improved Regret Bound and Experience Replay in Regularized Policy Iteration
• IROS 2019 Lifelong Robotic Vision Challenge -- Lifelong Object Recognition Report
• Variational Auto-Regressive Gaussian Processes for Continual Learning
• Disentangle-based Continual Graph Representation Learning
• Stabilizing Elastic Weight Consolidation method in practical ML tasks and using weight importances for neural network pruning
• AlterSGD: Finding Flat Minima for Continual Learning by Alternative Training
• Incremental Learning via Rate Reduction
• Tuned Compositional Feature Replays for Efficient Stream Learning
• TIE: A Framework for Embedding-based Incremental Temporal Knowledge Graph Completion
• A Procedural World Generation Framework for Systematic Evaluation of Continual Learning
• Continual Learning in Deep Networks: an Analysis of the Last Layer
• Explaining How Deep Neural Networks Forget by Deep Visualization
• Group and Exclusive Sparse Regularization-based Continual Learning of CNNs
• Towards Recognizing New Semantic Concepts in New Visual Domains
• Energy Aligning for Biased Models
• Subjectivity Learning Theory towards Artificial General Intelligence
• Deep Bayesian Unsupervised Lifelong Learning
• Memory and attention in deep learning
• Regularization-based Continual Learning for Fault Prediction in Lithium-Ion Batteries
• Reducing Catastrophic Forgetting in Modular Neural Networks by Dynamic Information Balancing
• An EM Framework for Online Incremental Learning of Semantic Segmentation
• DIODE: Dilatable Incremental Object Detection
• Selective Replay Enhances Learning in Online Continual Analogical Reasoning
• Learning Neural Models for Natural Language Processing in the Face of Distributional Shift
• Zero-shot task adaptation by homoiconic meta-mapping
• Complexity-aware Adaptive Training and Inference for Edge-Cloud Distributed AI Systems
• Data Summarization via Bilevel Optimization
• TyXe: Pyro-based Bayesian neural nets for Pytorch
• Bilevel Continual Learning
• Unsupervised Domain Expansion from Multiple Sources
• Generative Feature Replay with Orthogonal Weight Modification for Continual Learning
• SERIL: Noise Adaptive Speech Enhancement using Regularization-based Incremental Learning
• How do Quadratic Regularizers Prevent Catastrophic Forgetting: The Role of Interpolation
• Task-Projected Hyperdimensional Computing for Multi-Task Learning
• Online Reinforcement Learning Control by Direct Heuristic Dynamic Programming: from Time-Driven to Event-Driven
• Continuous Learning for Large-scale Personalized Domain Classification
• FFNB: Forgetting-Free Neural Blocks for Deep Continual Visual Learning
• Defeating Catastrophic Forgetting via Enhanced Orthogonal Weights Modification
• TAG: Task-based Accumulated Gradients for Lifelong learning
• A Biologically Plausible Audio-Visual Integration Model for Continual Learning
• Target Layer Regularization for Continual Learning Using Cramer-Wold Generator
• ZS-IL: Looking Back on Learned Experiences For Zero-Shot Incremental Learning
• Does the Adam Optimizer Exacerbate Catastrophic Forgetting?
• Adversarial Incremental Learning
• Residual Continual Learning
• Exploring the Challenges towards Lifelong Fact Learning
• Total Recall: a Customized Continual Learning Method for Neural Semantic Parsers
• Unsupervised Spiking Instance Segmentation on Event Data using STDP
• Lifelong Learning from Event-based Data
• Schematic Memory Persistence and Transience for Efficient and Robust Continual Learning
• Continual learning using hash-routed convolutional neural networks
• Incremental Class Learning using Variational Autoencoders with Similarity Learning
• Dendritic Self-Organizing Maps for Continual Learning
• Mixture-of-Variational-Experts for Continual Learning
• A Study on Efficiency in Continual Learning Inspired by Human Learning
• Do Not Forget to Attend to Uncertainty while Mitigating Catastrophic Forgetting
• Association: Remind Your GAN not to Forget
• Deep Virtual Networks for Memory Efficient Inference of Multiple Tasks
• Online Continual Learning in Image Classification: An Empirical Survey
• Dynamic VAEs with Generative Replay for Continual Zero-shot Learning
• Continual Learning of Generative Models with Limited Data: From Wasserstein-1 Barycenter to Adaptive Coalescence
• Lifelong Learning Without a Task Oracle
• Adaptive Neural Architectures for Recommender Systems
• Knowledge-Adaptation Priors
• Split-and-Bridge: Adaptable Class Incremental Learning within a Single Neural Network
• Shared and Private VAEs with Generative Replay for Continual Learning
• OvA-INN: Continual Learning with Invertible Neural Networks
• Lifelong Mixture of Variational Autoencoders
• IB-DRR: Incremental Learning with Information-Back Discrete Representation Replay
• Compression-aware Continual Learning using Singular Value Decomposition
• Class-incremental Learning with Rectified Feature-Graph Preservation
• MyMigrationBot: A Cloud-based Facebook Social Chatbot for Migrant Populations
• Better Knowledge Retention through Metric Learning
• FoCL: Feature-Oriented Continual Learning for Generative Models
• Condensed Composite Memory Continual Learning
• Continual learning under domain transfer with sparse synaptic bursting
• Adaptive Explainable Continual Learning Framework for Regression Problems with Focus on Power Forecasts
• Brain-inspired feature exaggeration in generative replay for continual learning