Sequoia: A Software Framework to Unify Continual Learning Research

Abstract

The field of Continual Learning (CL) seeks to develop algorithms that accumulate knowledge and skills over time through interaction with non-stationary environments. In practice, a plethora of evaluation procedures (settings) and algorithmic solutions (methods) exist, each with their own potentially disjoint set of assumptions. This variety makes measuring progress in CL difficult. We propose a taxonomy of settings, where each setting is described as a set of assumptions. A tree-shaped hierarchy emerges from this view, where more general settings become the parents of those with more restrictive assumptions. This makes it possible to use inheritance to share and reuse research, as developing a method for a given setting also makes it directly applicable onto any of its children. We instantiate this idea as a publicly available software framework called Sequoia, which features a wide variety of settings from both the Continual Supervised Learning (CSL) and Continual Reinforcement Learning (CRL) domains. Sequoia also includes a growing suite of methods which are easy to extend and customize, in addition to more specialized methods from external libraries. We hope that this new paradigm and its first implementation can help unify and accelerate research in CL. You can help us grow the tree by visiting www.github.com/lebrice/Sequoia.

References in corpus (23)

• Continuous control with deep reinforcement learning
• Overcoming catastrophic forgetting in neural networks
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
• A continual learning survey: Defying forgetting in classification tasks
• Continual Learning Through Synaptic Intelligence
• PathNet: Evolution Channels Gradient Descent in Super Neural Networks
• Efficient Lifelong Learning with A-GEM
• Three scenarios for continual learning
• Deep Reinforcement Learning that Matters
• Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning
• Overcoming catastrophic forgetting with hard attention to the task
• Towards Robust Evaluations of Continual Learning
• The Benefit of Multitask Representation Learning
• Generalization and Regularization in DQN
• Task Agnostic Continual Learning Using Online Variational Bayes
• Transfer in Deep Reinforcement Learning Using Successor Features and Generalised Policy Improvement
• DisCoRL: Continual Reinforcement Learning via Policy Distillation
• La-MAML: Look-ahead Meta Learning for Continual Learning
• Deep Reinforcement Learning amidst Lifelong Non-Stationarity
• Continuum: Simple Management of Complex Continual Learning Scenarios
• Continual Reinforcement Learning with Multi-Timescale Replay
• Continual Learning Using Bayesian Neural Networks

Cited by in corpus (2)

• CORA: Benchmarks, Baselines, and Metrics as a Platform for Continual Reinforcement Learning Agents
• CLEVA-Compass: A Continual Learning EValuation Assessment Compass to Promote Research Transparency and Comparability