Continual Learning: Tackling Catastrophic Forgetting in Deep Neural Networks with Replay Processes

Abstract

Humans learn all their life long. They accumulate knowledge from a sequence of learning experiences and remember the essential concepts without forgetting what they have learned previously. Artificial neural networks struggle to learn similarly. They often rely on data rigorously preprocessed to learn solutions to specific problems such as classification or regression. In particular, they forget their past learning experiences if trained on new ones. Therefore, artificial neural networks are often inept to deal with real-life settings such as an autonomous-robot that has to learn on-line to adapt to new situations and overcome new problems without forgetting its past learning-experiences. Continual learning (CL) is a branch of machine learning addressing this type of problem. Continual algorithms are designed to accumulate and improve knowledge in a curriculum of learning-experiences without forgetting. In this thesis, we propose to explore continual algorithms with replay processes. Replay processes gather together rehearsal methods and generative replay methods. Generative Replay consists of regenerating past learning experiences with a generative model to remember them. Rehearsal consists of saving a core-set of samples from past learning experiences to rehearse them later. The replay processes make possible a compromise between optimizing the current learning objective and the past ones enabling learning without forgetting in sequences of tasks settings. We show that they are very promising methods for continual learning. Notably, they enable the re-evaluation of past data with new knowledge and the confrontation of data from different learning-experiences. We demonstrate their ability to learn continually through unsupervised learning, supervised learning and reinforcement learning tasks.

arXiv commentDoctoral Thesis Manuscript, Institut Polytechnique de Paris (2020)

References in corpus (68)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Deep Learning in Neural Networks: An Overview
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Auto-Encoding Variational Bayes
• Distilling the Knowledge in a Neural Network
• Conditional Generative Adversarial Nets
• Overcoming catastrophic forgetting in neural networks
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Playing Atari with Deep Reinforcement Learning
• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• Categorical Reparameterization with Gumbel-Softmax
• End to End Learning for Self-Driving Cars
• Trust Region Policy Optimization
• Neural Discrete Representation Learning
• Conditional Image Synthesis With Auxiliary Classifier GANs
• Neural Architecture Search: A Survey
• LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop
• Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence
• Theano: new features and speed improvements
• Dota 2 with Large Scale Deep Reinforcement Learning
• Continual Learning Through Synaptic Intelligence
• An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks
• BEGAN: Boundary Equilibrium Generative Adversarial Networks
• QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation
• PathNet: Evolution Channels Gradient Descent in Super Neural Networks
• Efficient Lifelong Learning with A-GEM
• Optimizing Neural Networks with Kronecker-factored Approximate Curvature
• The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scale
• FearNet: Brain-Inspired Model for Incremental Learning
• State Representation Learning for Control: An Overview
• Domain Adaptation for Visual Applications: A Comprehensive Survey
• Overcoming Catastrophic Forgetting by Incremental Moment Matching
• Learning by Playing - Solving Sparse Reward Tasks from Scratch
• Online Structured Laplace Approximations For Overcoming Catastrophic Forgetting
• Towards Robust Evaluations of Continual Learning
• Intrinsically Motivated Goal Exploration Processes with Automatic Curriculum Learning
• Less-forgetting Learning in Deep Neural Networks
• Transfer from Simulation to Real World through Learning Deep Inverse Dynamics Model
• Classification Accuracy Score for Conditional Generative Models
• Deep Generative Dual Memory Network for Continual Learning
• Lifelong Generative Modeling
• Sim-to-Real Robot Learning from Pixels with Progressive Nets
• Embodied Evolution in Collective Robotics: A Review
• Continual Learning in Generative Adversarial Nets
• Sim-to-Real Reinforcement Learning for Deformable Object Manipulation
• Incremental Classifier Learning with Generative Adversarial Networks
• Loss is its own Reward: Self-Supervision for Reinforcement Learning
• Neurogenesis Deep Learning
• A comprehensive, application-oriented study of catastrophic forgetting in DNNs
• GEP-PG: Decoupling Exploration and Exploitation in Deep Reinforcement Learning Algorithms
• Life-Long Disentangled Representation Learning with Cross-Domain Latent Homologies
• Learning to Continually Learn
• Curiosity Driven Exploration of Learned Disentangled Goal Spaces
• Diffusion-based neuromodulation can eliminate catastrophic forgetting in simple neural networks
• Online Learned Continual Compression with Adaptive Quantization Modules
• Unicorn: Continual Learning with a Universal, Off-policy Agent
• From exploration to control: learning object manipulation skills through novelty search and local adaptation
• Computational Theories of Curiosity-Driven Learning
• Memory-based Parameter Adaptation
• Dataset Distillation
• Active Long Term Memory Networks
• Are We Ready for Service Robots? The OpenLORIS-Scene Datasets for Lifelong SLAM
• Regularization Shortcomings for Continual Learning
• S-RL Toolbox: Environments, Datasets and Evaluation Metrics for State Representation Learning
• Continual Learning in Neural Networks
• Distillation Techniques for Pseudo-rehearsal Based Incremental Learning
• Towards Lifelong Self-Supervision: A Deep Learning Direction for Robotics

Cited by in corpus (3)

• Continuum: Simple Management of Complex Continual Learning Scenarios
• Understanding Continual Learning Settings with Data Distribution Drift Analysis
• Learning to Transfer: A Foliated Theory