RLlib: Abstractions for Distributed Reinforcement Learning

Abstract

Reinforcement learning (RL) algorithms involve the deep nesting of highly irregular computation patterns, each of which typically exhibits opportunities for distributed computation. We argue for distributing RL components in a composable way by adapting algorithms for top-down hierarchical control, thereby encapsulating parallelism and resource requirements within short-running compute tasks. We demonstrate the benefits of this principle through RLlib: a library that provides scalable software primitives for RL. These primitives enable a broad range of algorithms to be implemented with high performance, scalability, and substantial code reuse. RLlib is available at https://rllib.io/.

arXiv commentPublished in the International Conference on Machine Learning (ICML 2018), 10 pages

Cited by in corpus (44)

• BigDL: A Distributed Deep Learning Framework for Big Data
• RLOC: Terrain-Aware Legged Locomotion using Reinforcement Learning and Optimal Control
• DRLE: Decentralized Reinforcement Learning at the Edge for Traffic Light Control in the IoV
• CityLearn v2: Energy-flexible, resilient, occupant-centric, and carbon-aware management of grid-interactive communities
• Unified Automatic Control of Vehicular Systems with Reinforcement Learning
• Colmena: Scalable Machine-Learning-Based Steering of Ensemble Simulations for High Performance Computing
• Distributed Deep Reinforcement Learning: A Survey and A Multi-Player Multi-Agent Learning Toolbox
• learn2learn: A Library for Meta-Learning Research
• Curriculum-based Reinforcement Learning for Distribution System Critical Load Restoration
• DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Frames
• Deep reinforcement learning applied to an assembly sequence planning problem with user preferences
• Creating Valid Adversarial Examples of Malware
• MADRaS : Multi Agent Driving Simulator
• Sustainability of Data Center Digital Twins with Reinforcement Learning
• ABIDES-Gym: Gym Environments for Multi-Agent Discrete Event Simulation and Application to Financial Markets
• Towards Real-World Deployment of Reinforcement Learning for Traffic Signal Control
• Motion Planning by Reinforcement Learning for an Unmanned Aerial Vehicle in Virtual Open Space with Static Obstacles
• Reinforcement Learning for Market Making in a Multi-agent Dealer Market
• Reinforcement learning pulses for transmon qubit entangling gates
• HYPPO: A Surrogate-Based Multi-Level Parallelism Tool for Hyperparameter Optimization
• Indoor Path Planning for an Unmanned Aerial Vehicle via Curriculum Learning
• Optimizing Mixed Autonomy Traffic Flow With Decentralized Autonomous Vehicles and Multi-Agent RL
• schlably: A Python Framework for Deep Reinforcement Learning Based Scheduling Experiments
• Safety Aware Autonomous Path Planning Using Model Predictive Reinforcement Learning for Inland Waterways
• Entropy-Aware Model Initialization for Effective Exploration in Deep Reinforcement Learning
• Mixed Cooperative-Competitive Communication Using Multi-Agent Reinforcement Learning
• XDO: A Double Oracle Algorithm for Extensive-Form Games
• A Comparison of Model-Free and Model Predictive Control for Price Responsive Water Heaters
• Indoor Path Planning for Multiple Unmanned Aerial Vehicles via Curriculum Learning
• Reinforcement Learning reveals fundamental limits on the mixing of active particles
• PyDCM: Custom Data Center Models with Reinforcement Learning for Sustainability
• Optimization of a Triangular Delaunay Mesh Generator using Reinforcement Learning
• MARLIM: Multi-Agent Reinforcement Learning for Inventory Management
• Co-Training an Observer and an Evading Target
• Learning to Communicate with Reinforcement Learning for an Adaptive Traffic Control System
• MDP Playground: An Analysis and Debug Testbed for Reinforcement Learning
• Deep Reinforcement Learning in Electricity Generation Investment for the Minimization of Long-Term Carbon Emissions and Electricity Costs
• Woodpecker-DL: Accelerating Deep Neural Networks via Hardware-Aware Multifaceted Optimizations
• Reinforcement Learning with Neural Networks for Quantum Multiple Hypothesis Testing
• Learning to See via Epiretinal Implant Stimulation in silico with Model-Based Deep Reinforcement Learning
• Deep Reinforcement Learning for Field Development Optimization
• PantheonRL: A MARL Library for Dynamic Training Interactions
• High Throughput Training of Deep Surrogates from Large Ensemble Runs
• Knowledge-guided Open Attribute Value Extraction with Reinforcement Learning