Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning
arXiv:2109.11978
Abstract
In this work, we present and study a training set-up that achieves fast policy generation for real-world robotic tasks by using massive parallelism on a single workstation GPU. We analyze and discuss the impact of different training algorithm components in the massively parallel regime on the final policy performance and training times. In addition, we present a novel game-inspired curriculum that is well suited for training with thousands of simulated robots in parallel. We evaluate the approach by training the quadrupedal robot ANYmal to walk on challenging terrain. The parallel approach allows training policies for flat terrain in under four minutes, and in twenty minutes for uneven terrain. This represents a speedup of multiple orders of magnitude compared to previous work. Finally, we transfer the policies to the real robot to validate the approach. We open-source our training code to help accelerate further research in the field of learned legged locomotion.
arXiv commentCoRL 2021 Project website: : https://leggedrobotics.github.io/legged_gym/ Video: https://youtu.be/8sO7VS3q8d0
Cited by in corpus (10)
- Orbit: A Unified Simulation Framework for Interactive Robot Learning Environments
- RLOC: Terrain-Aware Legged Locomotion using Reinforcement Learning and Optimal Control
- QuestSim: Human Motion Tracking from Sparse Sensors with Simulated Avatars
- Versatile Multi-Contact Planning and Control for Legged Loco-Manipulation
- Robust High-speed Running for Quadruped Robots via Deep Reinforcement Learning
- RL + Model-based Control: Using On-demand Optimal Control to Learn Versatile Legged Locomotion
- ViTAL: Vision-Based Terrain-Aware Locomotion for Legged Robots
- Benchmarking Potential Based Rewards for Learning Humanoid Locomotion
- Imitating and Finetuning Model Predictive Control for Robust and Symmetric Quadrupedal Locomotion
- MARLIN: Soft Actor-Critic based Reinforcement Learning for Congestion Control in Real Networks