Online Continual Learning on Sequences

Abstract

Online continual learning (OCL) refers to the ability of a system to learn over time from a continuous stream of data without having to revisit previously encountered training samples. Learning continually in a single data pass is crucial for agents and robots operating in changing environments and required to acquire, fine-tune, and transfer increasingly complex representations from non-i.i.d. input distributions. Machine learning models that address OCL must alleviate \textit{catastrophic forgetting} in which hidden representations are disrupted or completely overwritten when learning from streams of novel input. In this chapter, we summarize and discuss recent deep learning models that address OCL on sequential input through the use (and combination) of synaptic regularization, structural plasticity, and experience replay. Different implementations of replay have been proposed that alleviate catastrophic forgetting in connectionists architectures via the re-occurrence of (latent representations of) input sequences and that functionally resemble mechanisms of hippocampal replay in the mammalian brain. Empirical evidence shows that architectures endowed with experience replay typically outperform architectures without in (online) incremental learning tasks.

arXiv commentL. Oneto et al. (eds.), Recent Trends in Learning From Data, Studies in Computational Intelligence 896

References in corpus (12)

• Overcoming catastrophic forgetting in neural networks
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop
• The Arcade Learning Environment: An Evaluation Platform for General Agents
• Continual Learning Through Synaptic Intelligence
• FearNet: Brain-Inspired Model for Incremental Learning
• Batch Renormalization: Towards Reducing Minibatch Dependence in Batch-Normalized Models
• Neural Programmer-Interpreters
• Don't forget, there is more than forgetting: new metrics for Continual Learning
• Lifelong Learning of Spatiotemporal Representations with Dual-Memory Recurrent Self-Organization
• Unicorn: Continual Learning with a Universal, Off-policy Agent
• On the role of neurogenesis in overcoming catastrophic forgetting

Cited by in corpus (4)

• Evaluating Online Continual Learning with CALM
• Dynamic Nested Hierarchies: Pioneering Self-Evolution in Machine Learning Architectures for Lifelong Intelligence
• Don't Look Back in Anger: MAGIC Net for Streaming Continual Learning with Temporal Dependence
• Online Continual Learning in Image Classification: An Empirical Survey