CORA: Benchmarks, Baselines, and Metrics as a Platform for Continual Reinforcement Learning Agents

Abstract

Progress in continual reinforcement learning has been limited due to several barriers to entry: missing code, high compute requirements, and a lack of suitable benchmarks. In this work, we present CORA, a platform for Continual Reinforcement Learning Agents that provides benchmarks, baselines, and metrics in a single code package. The benchmarks we provide are designed to evaluate different aspects of the continual RL challenge, such as catastrophic forgetting, plasticity, ability to generalize, and sample-efficient learning. Three of the benchmarks utilize video game environments (Atari, Procgen, NetHack). The fourth benchmark, CHORES, consists of four different task sequences in a visually realistic home simulator, drawn from a diverse set of task and scene parameters. To compare continual RL methods on these benchmarks, we prepare three metrics in CORA: Continual Evaluation, Isolated Forgetting, and Zero-Shot Forward Transfer. Finally, CORA includes a set of performant, open-source baselines of existing algorithms for researchers to use and expand on. We release CORA and hope that the continual RL community can benefit from our contributions, to accelerate the development of new continual RL algorithms.

Repository available at https://github.com/AGI-Labs/continual_rl

References in corpus (15)

• Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm
• StarCraft II: A New Challenge for Reinforcement Learning
• DeepMind Control Suite
• robosuite: A Modular Simulation Framework and Benchmark for Robot Learning
• MINOS: Multimodal Indoor Simulator for Navigation in Complex Environments
• Rearrangement: A Challenge for Embodied AI
• HoME: a Household Multimodal Environment
• iGibson 1.0: a Simulation Environment for Interactive Tasks in Large Realistic Scenes
• VRKitchen: an Interactive 3D Virtual Environment for Task-oriented Learning
• TorchBeast: A PyTorch Platform for Distributed RL
• Habitat 2.0: Training Home Assistants to Rearrange their Habitat
• Continual World: A Robotic Benchmark For Continual Reinforcement Learning
• MiniHack the Planet: A Sandbox for Open-Ended Reinforcement Learning Research
• Jelly Bean World: A Testbed for Never-Ending Learning
• Sequoia: A Software Framework to Unify Continual Learning Research

Cited by in corpus (3)

• A Survey of Zero-shot Generalisation in Deep Reinforcement Learning
• A Domain-Agnostic Approach for Characterization of Lifelong Learning Systems
• A Study of Continual Learning Methods for Q-Learning