PathNet: Evolution Channels Gradient Descent in Super Neural Networks

Abstract

For artificial general intelligence (AGI) it would be efficient if multiple users trained the same giant neural network, permitting parameter reuse, without catastrophic forgetting. PathNet is a first step in this direction. It is a neural network algorithm that uses agents embedded in the neural network whose task is to discover which parts of the network to re-use for new tasks. Agents are pathways (views) through the network which determine the subset of parameters that are used and updated by the forwards and backwards passes of the backpropogation algorithm. During learning, a tournament selection genetic algorithm is used to select pathways through the neural network for replication and mutation. Pathway fitness is the performance of that pathway measured according to a cost function. We demonstrate successful transfer learning; fixing the parameters along a path learned on task A and re-evolving a new population of paths for task B, allows task B to be learned faster than it could be learned from scratch or after fine-tuning. Paths evolved on task B re-use parts of the optimal path evolved on task A. Positive transfer was demonstrated for binary MNIST, CIFAR, and SVHN supervised learning classification tasks, and a set of Atari and Labyrinth reinforcement learning tasks, suggesting PathNets have general applicability for neural network training. Finally, PathNet also significantly improves the robustness to hyperparameter choices of a parallel asynchronous reinforcement learning algorithm (A3C).

References in corpus (6)

• Distilling the Knowledge in a Neural Network
• Asynchronous Methods for Deep Reinforcement Learning
• Deep Exploration via Bootstrapped DQN
• Convolutional Neural Fabrics
• Net2Net: Accelerating Learning via Knowledge Transfer
• Swapout: Learning an ensemble of deep architectures

Cited by in corpus (186)

• A continual learning survey: Defying forgetting in classification tasks
• Three scenarios for continual learning
• Multi-Task Learning with Deep Neural Networks: A Survey
• The Natural Language Decathlon: Multitask Learning as Question Answering
• FearNet: Brain-Inspired Model for Incremental Learning
• Alleviating catastrophic forgetting using context-dependent gating and synaptic stabilization
• Overcoming catastrophic forgetting with hard attention to the task
• Reinforced Continual Learning
• Generative replay with feedback connections as a general strategy for continual learning
• Towards Robust Evaluations of Continual Learning
• Transfer Learning in Deep Reinforcement Learning: A Survey
• Incremental Learning for Semantic Segmentation of Large-Scale Remote Sensing Data
• Deep Generative Dual Memory Network for Continual Learning
• Born to Learn: the Inspiration, Progress, and Future of Evolved Plastic Artificial Neural Networks
• Federated Continual Learning via Knowledge Fusion: A Survey
• LAMOL: LAnguage MOdeling for Lifelong Language Learning
• Gradient Surgery for Multi-Task Learning
• Generative and Discriminative Text Classification with Recurrent Neural Networks
• Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interference
• Neural Episodic Control
• Continual Learning via Neural Pruning
• Incorporating Neuro-Inspired Adaptability for Continual Learning in Artificial Intelligence
• A Closer Look at Accuracy vs. Robustness
• Gotta Learn Fast: A New Benchmark for Generalization in RL
• Learn to Grow: A Continual Structure Learning Framework for Overcoming Catastrophic Forgetting
• Selfless Sequential Learning
• Recurrent Independent Mechanisms
• Learning to Continually Learn
• SpaceNet: Make Free Space For Continual Learning
• DualNet: Continual Learning, Fast and Slow
• Transfer Learning for Related Reinforcement Learning Tasks via Image-to-Image Translation
• MaskConnect: Connectivity Learning by Gradient Descent
• Domain-Incremental Continual Learning for Mitigating Bias in Facial Expression and Action Unit Recognition
• On Characterizing the Capacity of Neural Networks using Algebraic Topology
• Continual Deep Learning by Functional Regularisation of Memorable Past
• Overcoming Long-term Catastrophic Forgetting through Adversarial Neural Pruning and Synaptic Consolidation
• Continual Learning of a Mixed Sequence of Similar and Dissimilar Tasks
• Transferable Multi-Domain State Generator for Task-Oriented Dialogue Systems
• DER: Dynamically Expandable Representation for Class Incremental Learning
• Learning Abstract Options
• Ray Interference: a Source of Plateaus in Deep Reinforcement Learning
• Routing Networks and the Challenges of Modular and Compositional Computation
• Transfer Learning for Future Wireless Networks: A Comprehensive Survey
• Continual Semi-Supervised Learning through Contrastive Interpolation Consistency
• DisCoRL: Continual Reinforcement Learning via Policy Distillation
• Piggyback: Adapting a Single Network to Multiple Tasks by Learning to Mask Weights
• MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scale
• Controlled Forgetting: Targeted Stimulation and Dopaminergic Plasticity Modulation for Unsupervised Lifelong Learning in Spiking Neural Networks
• An Investigation of Replay-based Approaches for Continual Learning
• Modular meta-learning
• Continual Learning in Low-rank Orthogonal Subspaces
• Can Transfer Neuroevolution Tractably Solve Your Differential Equations?
• Efficient Continual Learning with Modular Networks and Task-Driven Priors
• Continual Learning with Adaptive Weights (CLAW)
• Pseudo-task Augmentation: From Deep Multitask Learning to Intratask Sharing---and Back
• Automatic Design of Artificial Neural Networks for Gamma-Ray Detection
• Toward Continual Learning for Conversational Agents
• Uncertainty-guided Continual Learning with Bayesian Neural Networks
• Connectivity Learning in Multi-Branch Networks
• Lifelong Object Detection
• Quantitative Phase Imaging and Artificial Intelligence: A Review
• AdaptCL: Adaptive Continual Learning for Tackling Heterogeneity in Sequential Datasets
• Replay in Deep Learning: Current Approaches and Missing Biological Elements
• Continual Reinforcement Learning with Complex Synapses
• Preserving Earlier Knowledge in Continual Learning with the Help of All Previous Feature Extractors
• Using Adapters to Overcome Catastrophic Forgetting in End-to-End Automatic Speech Recognition
• REMIND Your Neural Network to Prevent Catastrophic Forgetting
• Structural Compression of Convolutional Neural Networks
• Beyond Shared Hierarchies: Deep Multitask Learning through Soft Layer Ordering
• Catastrophic Interference in Reinforcement Learning: A Solution Based on Context Division and Knowledge Distillation
• Continual Classification Learning Using Generative Models
• An Adaptive Random Path Selection Approach for Incremental Learning
• Coordination Among Neural Modules Through a Shared Global Workspace
• Meta-learning curiosity algorithms
• Meta-Learning with Sparse Experience Replay for Lifelong Language Learning
• Generalized Variational Continual Learning
• Object Files and Schemata: Factorizing Declarative and Procedural Knowledge in Dynamical Systems
• Scheduling Computation Graphs of Deep Learning Models on Manycore CPUs
• Discrete-Valued Neural Communication
• Continual Learning in Neural Networks
• Are Neural Nets Modular? Inspecting Functional Modularity Through Differentiable Weight Masks
• Sentence Embedding Alignment for Lifelong Relation Extraction
• Lifelong Machine Learning with Deep Streaming Linear Discriminant Analysis
• Leveraging Old Knowledge to Continually Learn New Classes in Medical Images
• Feature Partitioning for Efficient Multi-Task Architectures
• Distillation Techniques for Pseudo-rehearsal Based Incremental Learning
• Tackling Catastrophic Forgetting and Background Shift in Continual Semantic Segmentation
• Lifelong Learning of Compositional Structures
• Sample-Efficient Reinforcement Learning through Transfer and Architectural Priors
• Autonomous Deep Learning: A Genetic DCNN Designer for Image Classification
• Learning to Multi-Task by Active Sampling
• Revisiting Spatial Invariance with Low-Rank Local Connectivity
• Encoders and Ensembles for Task-Free Continual Learning
• Continual Learning with Knowledge Transfer for Sentiment Classification
• Dynamic Inference with Neural Interpreters
• Learning Online Visual Invariances for Novel Objects via Supervised and Self-Supervised Training
• Modularity in Deep Learning: A Survey
• Transformers with Competitive Ensembles of Independent Mechanisms
• Online Structured Meta-learning
• Evolutionary Architecture Search For Deep Multitask Networks
• Self-Supervised Learning Aided Class-Incremental Lifelong Learning
• Flexible Multi-task Networks by Learning Parameter Allocation
• Memory Efficient Experience Replay for Streaming Learning
• Powerpropagation: A sparsity inducing weight reparameterisation
• Evaluating Online Continual Learning with CALM
• Fast Neural Architecture Construction using EnvelopeNets
• Understanding Continual Learning Settings with Data Distribution Drift Analysis
• Lifelong Learning of Hate Speech Classification on Social Media
• A Study of Continual Learning Methods for Q-Learning
• Continual Learning in Deep Neural Network by Using a Kalman Optimiser
• Energy-Based Models for Continual Learning
• NEUZZ: Efficient Fuzzing with Neural Program Smoothing
• How to Evaluate the Next System: Automatic Dialogue Evaluation from the Perspective of Continual Learning
• Hierarchical semantic segmentation using modular convolutional neural networks
• Unifying Regularisation Methods for Continual Learning
• Bayesian Structure Adaptation for Continual Learning
• Overcoming Catastrophic Forgetting by Neuron-level Plasticity Control
• AE-Net: Autonomous Evolution Image Fusion Method Inspired by Human Cognitive Mechanism
• Thalamocortical contribution to solving credit assignment in neural systems
• Simple Genetic Operators are Universal Approximators of Probability Distributions (and other Advantages of Expressive Encodings)
• Sequoia: A Software Framework to Unify Continual Learning Research
• Wide Neural Networks Forget Less Catastrophically
• Modality-Transferable Emotion Embeddings for Low-Resource Multimodal Emotion Recognition
• Overcoming Catastrophic Forgetting by Generative Regularization
• Rethinking Experience Replay: a Bag of Tricks for Continual Learning
• Continual Learning: Tackling Catastrophic Forgetting in Deep Neural Networks with Replay Processes
• Fisher Task Distance and Its Application in Neural Architecture Search
• Continual Learning via Bit-Level Information Preserving
• Review, Analysis and Design of a Comprehensive Deep Reinforcement Learning Framework
• Optimizing Reusable Knowledge for Continual Learning via Metalearning
• On Catastrophic Interference in Atari 2600 Games
• Lifelong Vehicle Trajectory Prediction Framework Based on Generative Replay
• Reviewing continual learning from the perspective of human-level intelligence
• A Conceptual Bio-Inspired Framework for the Evolution of Artificial General Intelligence
• OER: Offline Experience Replay for Continual Offline Reinforcement Learning
• Overcoming Catastrophic Forgetting by Soft Parameter Pruning
• Continual Learning in the Presence of Repetition
• GROWN: GRow Only When Necessary for Continual Learning
• Sequential Targeting: an incremental learning approach for data imbalance in text classification
• Bilevel Continual Learning
• Frosting Weights for Better Continual Training
• Generative Memory for Lifelong Reinforcement Learning
• Regularize, Expand and Compress: Multi-task based Lifelong Learning via NonExpansive AutoML
• Multi-task Learning with Gradient Guided Policy Specialization
• Continual Reinforcement Learning with Diversity Exploration and Adversarial Self-Correction
• Continual Learning in Task-Oriented Dialogue Systems
• Graph-Based Continual Learning
• An Orthogonal-SGD based Learning Approach for MIMO Detection under Multiple Channel Models
• Neural Function Modules with Sparse Arguments: A Dynamic Approach to Integrating Information across Layers
• Closed-Loop Memory GAN for Continual Learning
• Learning to generate classifiers
• Learning to Remember from a Multi-Task Teacher
• Power Law in Sparsified Deep Neural Networks
• Lifelong Learning with Searchable Extension Units
• CatNet: Class Incremental 3D ConvNets for Lifelong Egocentric Gesture Recognition
• Zero-shot task adaptation by homoiconic meta-mapping
• DIODE: Dilatable Incremental Object Detection
• Tuned Compositional Feature Replays for Efficient Stream Learning
• Software/Hardware Co-design for Multi-modal Multi-task Learning in Autonomous Systems
• Modular meta-learning in abstract graph networks for combinatorial generalization
• Task Affinity with Maximum Bipartite Matching in Few-Shot Learning
• Recursive Sketches for Modular Deep Learning
• Class Incremental Online Streaming Learning
• Collaborative Group Learning
• General AI Challenge - Round One: Gradual Learning
• Catastrophic Importance of Catastrophic Forgetting
• Modular Continual Learning in a Unified Visual Environment
• Eden: A Unified Environment Framework for Booming Reinforcement Learning Algorithms
• Multitask Evolution with Cartesian Genetic Programming
• Online Continual Learning in Image Classification: An Empirical Survey
• SAFE: Spectral Evolution Analysis Feature Extraction for Non-Stationary Time Series Prediction
• Sequential Coordination of Deep Models for Learning Visual Arithmetic
• Discriminative Distillation to Reduce Class Confusion in Continual Learning
• Mixture-of-Variational-Experts for Continual Learning
• Incremental Class Learning using Variational Autoencoders with Similarity Learning
• A Biologically Plausible Audio-Visual Integration Model for Continual Learning
• Continuous Learning for Large-scale Personalized Domain Classification
• Exploring the Challenges towards Lifelong Fact Learning
• Continual learning under domain transfer with sparse synaptic bursting
• Continual Learning for Image-Based Camera Localization
• Adaptive Explainable Continual Learning Framework for Regression Problems with Focus on Power Forecasts
• On Anytime Learning at Macroscale
• Shared and Private VAEs with Generative Replay for Continual Learning
• Strongly-Typed Agents are Guaranteed to Interact Safely
• Enhancing Reinforcement Learning with discrete interfaces to learn the Dyck Language
• RSAC: Regularized Subspace Approximation Classifier for Lightweight Continuous Learning