GPU-Accelerated Robotic Simulation for Distributed Reinforcement Learning
arXiv:1810.05762
Abstract
Most Deep Reinforcement Learning (Deep RL) algorithms require a prohibitively large number of training samples for learning complex tasks. Many recent works on speeding up Deep RL have focused on distributed training and simulation. While distributed training is often done on the GPU, simulation is not. In this work, we propose using GPU-accelerated RL simulations as an alternative to CPU ones. Using NVIDIA Flex, a GPU-based physics engine, we show promising speed-ups of learning various continuous-control, locomotion tasks. With one GPU and CPU core, we are able to train the Humanoid running task in less than 20 minutes, using 10-1000x fewer CPU cores than previous works. We also demonstrate the scalability of our simulator to multi-GPU settings to train more challenging locomotion tasks.
arXiv commentAccepted and to appear at the Conference on Robot Learning (CoRL) 2018
References in corpus (16)
- Evolution Strategies as a Scalable Alternative to Reinforcement Learning
- Benchmarking Deep Reinforcement Learning for Continuous Control
- Emergence of Locomotion Behaviours in Rich Environments
- IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures
- Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning
- DeepMind Control Suite
- Self-Normalizing Neural Networks
- Horovod: fast and easy distributed deep learning in TensorFlow
- Distributed Prioritized Experience Replay
- Massively Parallel Methods for Deep Reinforcement Learning
- Learning Dexterous In-Hand Manipulation
- Simple random search provides a competitive approach to reinforcement learning
- Sim-to-Real: Learning Agile Locomotion For Quadruped Robots
- Accelerated Methods for Deep Reinforcement Learning
- GPU accelerated Monte Carlo simulation of Brownian motors dynamics with CUDA
- GPU-accelerated Red Blood Cells Simulations with Transport Dissipative Particle Dynamics