Progress & Compress: A scalable framework for continual learning

Abstract

We introduce a conceptually simple and scalable framework for continual learning domains where tasks are learned sequentially. Our method is constant in the number of parameters and is designed to preserve performance on previously encountered tasks while accelerating learning progress on subsequent problems. This is achieved by training a network with two components: A knowledge base, capable of solving previously encountered problems, which is connected to an active column that is employed to efficiently learn the current task. After learning a new task, the active column is distilled into the knowledge base, taking care to protect any previously acquired skills. This cycle of active learning (progression) followed by consolidation (compression) requires no architecture growth, no access to or storing of previous data or tasks, and no task-specific parameters. We demonstrate the progress & compress approach on sequential classification of handwritten alphabets as well as two reinforcement learning domains: Atari games and 3D maze navigation.

arXiv commentAccepted at ICML 2018

Cited by in corpus (66)

• A continual learning survey: Defying forgetting in classification tasks
• Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence
• A Survey on Anomaly Detection for Technical Systems using LSTM Networks
• Experience Replay for Continual Learning
• Continual Learning of Context-dependent Processing in Neural Networks
• Ditto: Fair and Robust Federated Learning Through Personalization
• Continual Learning for Recurrent Neural Networks: an Empirical Evaluation
• Variational Federated Multi-Task Learning
• Federated Continual Learning via Knowledge Fusion: A Survey
• Lifelong Generative Modeling
• Pseudo-Rehearsal: Achieving Deep Reinforcement Learning without Catastrophic Forgetting
• A Neural Dirichlet Process Mixture Model for Task-Free Continual Learning
• Incorporating Neuro-Inspired Adaptability for Continual Learning in Artificial Intelligence
• SpaceNet: Make Free Space For Continual Learning
• Domain-Incremental Continual Learning for Mitigating Bias in Facial Expression and Action Unit Recognition
• A Bayesian Federated Learning Framework with Online Laplace Approximation
• Overcoming Long-term Catastrophic Forgetting through Adversarial Neural Pruning and Synaptic Consolidation
• Efficient Continual Learning in Neural Networks with Embedding Regularization
• Continual Learning and Catastrophic Forgetting
• Inexact-ADMM Based Federated Meta-Learning for Fast and Continual Edge Learning
• CLeaR: An Adaptive Continual Learning Framework for Regression Tasks
• Learning offline: memory replay in biological and artificial reinforcement learning
• Orthogonal Gradient Descent for Continual Learning
• Continual Semi-Supervised Learning through Contrastive Interpolation Consistency
• Class-incremental Learning with Pre-allocated Fixed Classifiers
• Continuous Domain Adaptation with Variational Domain-Agnostic Feature Replay
• Class-incremental Learning via Deep Model Consolidation
• Continual Learning with Adaptive Weights (CLAW)
• RATT: Recurrent Attention to Transient Tasks for Continual Image Captioning
• Continual Learning via Inter-Task Synaptic Mapping
• A Dirichlet Process Mixture of Robust Task Models for Scalable Lifelong Reinforcement Learning
• Continual learning of longitudinal health records
• Complementary Calibration: Boosting General Continual Learning with Collaborative Distillation and Self-Supervision
• Regularization Shortcomings for Continual Learning
• A Domain-Agnostic Approach for Characterization of Lifelong Learning Systems
• Accented Speech Recognition: A Survey
• GP-Tree: A Gaussian Process Classifier for Few-Shot Incremental Learning
• Lifelong Policy Gradient Learning of Factored Policies for Faster Training Without Forgetting
• Continual BERT: Continual Learning for Adaptive Extractive Summarization of COVID-19 Literature
• CORA: Benchmarks, Baselines, and Metrics as a Platform for Continual Reinforcement Learning Agents
• Lifelong Robotic Reinforcement Learning by Retaining Experiences
• A Self-Commissioning Edge Computing Method for Data-Driven Anomaly Detection in Power Electronic Systems
• Continuous Coordination As a Realistic Scenario for Lifelong Learning
• On Sequential Bayesian Inference for Continual Learning
• Understanding Continual Learning Settings with Data Distribution Drift Analysis
• OpenLORIS-Object: A Robotic Vision Dataset and Benchmark for Lifelong Deep Learning
• Continual Learning via Online Leverage Score Sampling
• Bayesian Structure Adaptation for Continual Learning
• Few-Shot Unsupervised Continual Learning through Meta-Examples
• Frosting Weights for Better Continual Training
• Rethinking Experience Replay: a Bag of Tricks for Continual Learning
• Review learning: Real world validation of privacy preserving continual learning across medical institutions
• GROWN: GRow Only When Necessary for Continual Learning
• SID: Incremental Learning for Anchor-Free Object Detection via Selective and Inter-Related Distillation
• Lifelong Reinforcement Learning with Temporal Logic Formulas and Reward Machines
• Complexity-aware Adaptive Training and Inference for Edge-Cloud Distributed AI Systems
• Visually Grounded Continual Language Learning with Selective Specialization
• Tuned Compositional Feature Replays for Efficient Stream Learning
• Continual Machine Reading Comprehension via Uncertainty-aware Fixed Memory and Adversarial Domain Adaptation
• Self-Net: Lifelong Learning via Continual Self-Modeling
• DIODE: Dilatable Incremental Object Detection
• Monitoring nonstationary processes based on recursive cointegration analysis and elastic weight consolidation
• SERIL: Noise Adaptive Speech Enhancement using Regularization-based Incremental Learning
• Online Continual Learning in Image Classification: An Empirical Survey
• Split-and-Bridge: Adaptable Class Incremental Learning within a Single Neural Network
• RSAC: Regularized Subspace Approximation Classifier for Lightweight Continuous Learning