SLM Lab: A Comprehensive Benchmark and Modular Software Framework for Reproducible Deep Reinforcement Learning

Abstract

We introduce SLM Lab, a software framework for reproducible reinforcement learning (RL) research. SLM Lab implements a number of popular RL algorithms, provides synchronous and asynchronous parallel experiment execution, hyperparameter search, and result analysis. RL algorithms in SLM Lab are implemented in a modular way such that differences in algorithm performance can be confidently ascribed to differences between algorithms, not between implementations. In this work we present the design choices behind SLM Lab and use it to produce a comprehensive single-codebase RL algorithm benchmark. In addition, as a consequence of SLM Lab's modular design, we introduce and evaluate a discrete-action variant of the Soft Actor-Critic algorithm (Haarnoja et al., 2018) and a hybrid synchronous/asynchronous training method for RL agents.

References in corpus (8)

• Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
• Soft Actor-Critic Algorithms and Applications
• The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables
• Tune: A Research Platform for Distributed Model Selection and Training
• Unity: A General Platform for Intelligent Agents
• Reproducibility of Benchmarked Deep Reinforcement Learning Tasks for Continuous Control
• Horizon: Facebook's Open Source Applied Reinforcement Learning Platform
• rlpyt: A Research Code Base for Deep Reinforcement Learning in PyTorch

Cited by in corpus (3)

• RLlib Flow: Distributed Reinforcement Learning is a Dataflow Problem
• Sample Efficient Ensemble Learning with Catalyst.RL
• The AI Arena: A Framework for Distributed Multi-Agent Reinforcement Learning