Progressive Neural Networks

Abstract

Learning to solve complex sequences of tasks--while both leveraging transfer and avoiding catastrophic forgetting--remains a key obstacle to achieving human-level intelligence. The progressive networks approach represents a step forward in this direction: they are immune to forgetting and can leverage prior knowledge via lateral connections to previously learned features. We evaluate this architecture extensively on a wide variety of reinforcement learning tasks (Atari and 3D maze games), and show that it outperforms common baselines based on pretraining and finetuning. Using a novel sensitivity measure, we demonstrate that transfer occurs at both low-level sensory and high-level control layers of the learned policy.

Cited by in corpus (86)

• Overcoming catastrophic forgetting in neural networks
• A Brief Survey of Deep Reinforcement Learning
• Deep Visual Domain Adaptation: A Survey
• A continual learning survey: Defying forgetting in classification tasks
• Deep Reinforcement Learning for Multi-Agent Systems: A Review of Challenges, Solutions and Applications
• Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence
• State Representation Learning for Control: An Overview
• Class-Incremental Learning: A Survey
• Alleviating catastrophic forgetting using context-dependent gating and synaptic stabilization
• Lifelong Federated Reinforcement Learning: A Learning Architecture for Navigation in Cloud Robotic Systems
• A Comprehensive Study of Class Incremental Learning Algorithms for Visual Tasks
• Incremental Learning for Semantic Segmentation of Large-Scale Remote Sensing Data
• Incremental Learning in Deep Convolutional Neural Networks Using Partial Network Sharing
• Continual Learning for Recurrent Neural Networks: an Empirical Evaluation
• Federated Continual Learning via Knowledge Fusion: A Survey
• A survey on GANs for computer vision: Recent research, analysis and taxonomy
• Lifelong Generative Modeling
• Pseudo-Rehearsal: Achieving Deep Reinforcement Learning without Catastrophic Forgetting
• Continual Learning in Sensor-based Human Activity Recognition: an Empirical Benchmark Analysis
• Lifelong Teacher-Student Network Learning
• Overcoming Long-term Catastrophic Forgetting through Adversarial Neural Pruning and Synaptic Consolidation
• Triple Memory Networks: a Brain-Inspired Method for Continual Learning
• AdaER: An Adaptive Experience Replay Approach for Continual Lifelong Learning
• Cognitively-Inspired Model for Incremental Learning Using a Few Examples
• Deep progressive reinforcement learning-based flexible resource scheduling framework for IRS and UAV-assisted MEC system
• Lifelong Machine Learning Potentials
• CLeaR: An Adaptive Continual Learning Framework for Regression Tasks
• Diffusion-based neuromodulation can eliminate catastrophic forgetting in simple neural networks
• Learning offline: memory replay in biological and artificial reinforcement learning
• Budget-Aware Adapters for Multi-Domain Learning
• Multimodal Parameter-Efficient Few-Shot Class Incremental Learning
• Dynamically Expandable Graph Convolution for Streaming Recommendation
• Online Hyperparameter Optimization for Class-Incremental Learning
• Memory Based Online Learning of Deep Representations from Video Streams
• Continual Learning with Gated Incremental Memories for sequential data processing
• Learning an Adaptive Meta Model-Generator for Incrementally Updating Recommender Systems
• Continual Learning via Inter-Task Synaptic Mapping
• A Dirichlet Process Mixture of Robust Task Models for Scalable Lifelong Reinforcement Learning
• Uncertainty-based Modulation for Lifelong Learning
• AdaptCL: Adaptive Continual Learning for Tackling Heterogeneity in Sequential Datasets
• Learning to Continuously Optimize Wireless Resource In Episodically Dynamic Environment
• Catastrophic Interference in Reinforcement Learning: A Solution Based on Context Division and Knowledge Distillation
• Learning Representations for New Sound Classes With Continual Self-Supervised Learning
• Beneficial Perturbation Network for designing general adaptive artificial intelligence systems
• Unsupervised Reinforcement Learning for Transferable Manipulation Skill Discovery
• Continual Horizontal Federated Learning for Heterogeneous Data
• Class Gradient Projection For Continual Learning
• Online Continual Learning via the Knowledge Invariant and Spread-out Properties
• A multifidelity approach to continual learning for physical systems
• Self-Attentional Credit Assignment for Transfer in Reinforcement Learning
• Learning more with the same effort: how randomization improves the robustness of a robotic deep reinforcement learning agent
• Procedural Content Generation via Knowledge Transformation (PCG-KT)
• Learning Online Visual Invariances for Novel Objects via Supervised and Self-Supervised Training
• Modularity in Deep Learning: A Survey
• Rationalizing Predictions by Adversarial Information Calibration
• Boosting Binary Masks for Multi-Domain Learning through Affine Transformations
• Preventing Catastrophic Forgetting in Continual Learning of New Natural Language Tasks
• All by Myself: Learning Individualized Competitive Behaviour with a Contrastive Reinforcement Learning optimization
• Towards Foundation Models and Few-Shot Parameter-Efficient Fine-Tuning for Volumetric Organ Segmentation
• Weight Averaging: A Simple Yet Effective Method to Overcome Catastrophic Forgetting in Automatic Speech Recognition
• Hessian Aware Low-Rank Perturbation for Order-Robust Continual Learning
• Continual Referring Expression Comprehension via Dual Modular Memorization
• A Study of Continual Learning Methods for Q-Learning
• Targeted Gradient Descent: A Novel Method for Convolutional Neural Networks Fine-tuning and Online-learning
• CUCL: Codebook for Unsupervised Continual Learning
• A Comparative Study of Calibration Methods for Imbalanced Class Incremental Learning
• DRILL: Dynamic Representations for Imbalanced Lifelong Learning
• Continual Learning in the Presence of Repetition
• Review learning: Real world validation of privacy preserving continual learning across medical institutions
• Fisher Task Distance and Its Application in Neural Architecture Search
• Frosting Weights for Better Continual Training
• Sim-to-Real Transfer via a Style-Identified Cycle Consistent Generative Adversarial Network: Zero-Shot Deployment on Robotic Manipulators through Visual Domain Adaptation
• Optimal Protocols for Continual Learning via Statistical Physics and Control Theory
• Continual Learning: Forget-free Winning Subnetworks for Video Representations
• Curriculum-Meta Learning for Order-Robust Continual Relation Extraction
• RECALL: Rehearsal-free Continual Learning for Object Classification
• A Deep Value-network Based Approach for Multi-Driver Order Dispatching
• Dynamic Continual Learning: Harnessing Parameter Uncertainty for Improved Network Adaptation
• DeCoR: Defy Knowledge Forgetting by Predicting Earlier Audio Codes
• Visually Grounded Continual Language Learning with Selective Specialization
• An Active Learning Framework for Efficient Robust Policy Search
• Continually Learn to Map Visual Concepts to Large Language Models in Resource-constrained Environments
• Toward industrial use of continual learning : new metrics proposal for class incremental learning
• Context selectivity with dynamic availability enables lifelong continual learning
• Weighted Ensemble Models Are Strong Continual Learners
• A Biologically Plausible Audio-Visual Integration Model for Continual Learning