Tonic: A Deep Reinforcement Learning Library for Fast Prototyping and Benchmarking

Abstract

Deep reinforcement learning has been one of the fastest growing fields of machine learning over the past years and numerous libraries have been open sourced to support research. However, most codebases have a steep learning curve or limited flexibility that do not satisfy a need for fast prototyping in fundamental research. This paper introduces Tonic, a Python library allowing researchers to quickly implement new ideas and measure their importance by providing: 1) general-purpose configurable modules 2) several baseline agents: A2C, TRPO, PPO, MPO, DDPG, D4PG, TD3 and SAC built with these modules 3) support for TensorFlow 2 and PyTorch 4) support for continuous-control environments from OpenAI Gym, DeepMind Control Suite and PyBullet 5) scripts to experiment in a reproducible way, plot results, and play with trained agents 6) a benchmark of the provided agents on 70 continuous-control tasks. Evaluation is performed in fair conditions with identical seeds, training and testing loops, while sharing general improvements such as non-terminal timeouts and observation normalization. Finally, to demonstrate how Tonic simplifies experimentation, a novel agent called TD4 is implemented and evaluated.

arXiv commentCode: https://github.com/fabiopardo/tonic

References in corpus (15)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Continuous control with deep reinforcement learning
• Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
• Trust Region Policy Optimization
• Dota 2 with Large Scale Deep Reinforcement Learning
• IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures
• DeepMind Control Suite
• Distributed Distributional Deterministic Policy Gradients
• Maximum a Posteriori Policy Optimisation
• Dopamine: A Research Framework for Deep Reinforcement Learning
• Acme: A Research Framework for Distributed Reinforcement Learning
• Discrete Sequential Prediction of Continuous Actions for Deep RL
• rlpyt: A Research Code Base for Deep Reinforcement Learning in PyTorch
• MushroomRL: Simplifying Reinforcement Learning Research
• Q-Learning in enormous action spaces via amortized approximate maximization

Cited by in corpus (4)

• d3rlpy: An Offline Deep Reinforcement Learning Library
• Evolving and Merging Hebbian Learning Rules: Increasing Generalization by Decreasing the Number of Rules
• Is Bang-Bang Control All You Need? Solving Continuous Control with Bernoulli Policies
• Learning to Represent Action Values as a Hypergraph on the Action Vertices