Meta Arcade: A Configurable Environment Suite for Meta-Learning

Abstract

Most approaches to deep reinforcement learning (DRL) attempt to solve a single task at a time. As a result, most existing research benchmarks consist of individual games or suites of games that have common interfaces but little overlap in their perceptual features, objectives, or reward structures. To facilitate research into knowledge transfer among trained agents (e.g. via multi-task and meta-learning), more environment suites that provide configurable tasks with enough commonality to be studied collectively are needed. In this paper we present Meta Arcade, a tool to easily define and configure custom 2D arcade games that share common visuals, state spaces, action spaces, game components, and scoring mechanisms. Meta Arcade differs from prior environments in that both task commonality and configurability are prioritized: entire sets of games can be constructed from common elements, and these elements are adjustable through exposed parameters. We include a suite of 24 predefined games that collectively illustrate the possibilities of this framework and discuss how these games can be configured for research applications. We provide several experiments that illustrate how Meta Arcade could be used, including single-task benchmarks of predefined games, sample curriculum-based approaches that change game parameters over a set schedule, and an exploration of transfer learning between games.

17 pages, 6 figures, 6 tables, extended version of an accepted paper to NeurIPS DRL Workshop 2021

References in corpus (12)

• Continuous control with deep reinforcement learning
• Trust Region Policy Optimization
• High-Dimensional Continuous Control Using Generalized Advantage Estimation
• StarCraft II: A New Challenge for Reinforcement Learning
• Unity: A General Platform for Intelligent Agents
• Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning
• A Study on Overfitting in Deep Reinforcement Learning
• Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World
• Quantifying Generalization in Reinforcement Learning
• Leveraging Procedural Generation to Benchmark Reinforcement Learning
• Gotta Learn Fast: A New Benchmark for Generalization in RL
• Accelerating Reinforcement Learning for Reaching using Continuous Curriculum Learning