Sample Efficient Ensemble Learning with Catalyst.RL

Abstract

We present Catalyst.RL, an open-source PyTorch framework for reproducible and sample efficient reinforcement learning (RL) research. Main features of Catalyst.RL include large-scale asynchronous distributed training, efficient implementations of various RL algorithms and auxiliary tricks, such as n-step returns, value distributions, hyperbolic reinforcement learning, etc. To demonstrate the effectiveness of Catalyst.RL, we applied it to a physics-based reinforcement learning challenge "NeurIPS 2019: Learn to Move -- Walk Around" with the objective to build a locomotion controller for a human musculoskeletal model. The environment is computationally expensive, has a high-dimensional continuous action space and is stochastic. Our team took the 2nd place, capitalizing on the ability of Catalyst.RL to train high-quality and sample-efficient RL agents in only a few hours of training time. The implementation along with experiments is open-sourced so results can be reproduced and novel ideas tried out.

arXiv admin note: substantial text overlap with arXiv:1903.00027

References in corpus (10)

• Neural Architecture Search with Reinforcement Learning
• Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm
• Parameter Space Noise for Exploration
• An Actor-Critic Algorithm for Sequence Prediction
• Reproducibility of Benchmarked Deep Reinforcement Learning Tasks for Continuous Control
• Dopamine: A Research Framework for Deep Reinforcement Learning
• Distributional Reinforcement Learning with Quantile Regression
• Hyperbolic Discounting and Learning over Multiple Horizons
• Learning to Run with Actor-Critic Ensemble
• SLM Lab: A Comprehensive Benchmark and Modular Software Framework for Reproducible Deep Reinforcement Learning