PathNet: Evolution Channels Gradient Descent in Super Neural Networks

Abstract

For artificial general intelligence (AGI) it would be efficient if multiple users trained the same giant neural network, permitting parameter reuse, without catastrophic forgetting. PathNet is a first step in this direction. It is a neural network algorithm that uses agents embedded in the neural network whose task is to discover which parts of the network to re-use for new tasks. Agents are pathways (views) through the network which determine the subset of parameters that are used and updated by the forwards and backwards passes of the backpropogation algorithm. During learning, a tournament selection genetic algorithm is used to select pathways through the neural network for replication and mutation. Pathway fitness is the performance of that pathway measured according to a cost function. We demonstrate successful transfer learning; fixing the parameters along a path learned on task A and re-evolving a new population of paths for task B, allows task B to be learned faster than it could be learned from scratch or after fine-tuning. Paths evolved on task B re-use parts of the optimal path evolved on task A. Positive transfer was demonstrated for binary MNIST, CIFAR, and SVHN supervised learning classification tasks, and a set of Atari and Labyrinth reinforcement learning tasks, suggesting PathNets have general applicability for neural network training. Finally, PathNet also significantly improves the robustness to hyperparameter choices of a parallel asynchronous reinforcement learning algorithm (A3C).

References in corpus (1)

• Distilling the Knowledge in a Neural Network

Cited by in corpus (34)

• Three scenarios for continual learning
• Multi-Task Learning with Deep Neural Networks: A Survey
• Neural Episodic Control
• Continual Learning via Neural Pruning
• Learn to Grow: A Continual Structure Learning Framework for Overcoming Catastrophic Forgetting
• Learning to Continually Learn
• Transferable Multi-Domain State Generator for Task-Oriented Dialogue Systems
• Ray Interference: a Source of Plateaus in Deep Reinforcement Learning
• Routing Networks and the Challenges of Modular and Compositional Computation
• DisCoRL: Continual Reinforcement Learning via Policy Distillation
• Toward Continual Learning for Conversational Agents
• Connectivity Learning in Multi-Branch Networks
• Lifelong Object Detection
• Continual Learning in Neural Networks
• Sentence Embedding Alignment for Lifelong Relation Extraction
• Sample-Efficient Reinforcement Learning through Transfer and Architectural Priors
• Continual Learning in Deep Neural Network by Using a Kalman Optimiser
• How to Evaluate the Next System: Automatic Dialogue Evaluation from the Perspective of Continual Learning
• Hierarchical semantic segmentation using modular convolutional neural networks
• AE-Net: Autonomous Evolution Image Fusion Method Inspired by Human Cognitive Mechanism
• Bayesian Structure Adaptation for Continual Learning
• Regularize, Expand and Compress: Multi-task based Lifelong Learning via NonExpansive AutoML
• Generative Memory for Lifelong Reinforcement Learning
• Frosting Weights for Better Continual Training
• Bilevel Continual Learning
• Learning to Remember from a Multi-Task Teacher
• An Orthogonal-SGD based Learning Approach for MIMO Detection under Multiple Channel Models
• Continual Reinforcement Learning with Diversity Exploration and Adversarial Self-Correction
• Modular meta-learning in abstract graph networks for combinatorial generalization
• Continuous Learning for Large-scale Personalized Domain Classification
• General AI Challenge - Round One: Gradual Learning
• RSAC: Regularized Subspace Approximation Classifier for Lightweight Continuous Learning
• Exploring the Challenges towards Lifelong Fact Learning
• Modular Continual Learning in a Unified Visual Environment