Learning agile and dynamic motor skills for legged robots
arXiv:1901.08652 · doi:10.1126/scirobotics.aau5872
Abstract
Legged robots pose one of the greatest challenges in robotics. Dynamic and agile maneuvers of animals cannot be imitated by existing methods that are crafted by humans. A compelling alternative is reinforcement learning, which requires minimal craftsmanship and promotes the natural evolution of a control policy. However, so far, reinforcement learning research for legged robots is mainly limited to simulation, and only few and comparably simple examples have been deployed on real systems. The primary reason is that training with real robots, particularly with dynamically balancing systems, is complicated and expensive. In the present work, we introduce a method for training a neural network policy in simulation and transferring it to a state-of-the-art legged system, thereby leveraging fast, automated, and cost-effective data generation schemes. The approach is applied to the ANYmal robot, a sophisticated medium-dog-sized quadrupedal system. Using policies trained in simulation, the quadrupedal machine achieves locomotion skills that go beyond what had been achieved with prior methods: ANYmal is capable of precisely and energy-efficiently following high-level body velocity commands, running faster than before, and recovering from falling even in complex configurations.
References in corpus (2)
Cited by in corpus (75)
- Learning robust perceptive locomotion for quadrupedal robots in the wild
- Solving Rubik's Cube with a Robot Hand
- How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learned
- Multi-expert learning of adaptive legged locomotion
- Concurrent Training of a Control Policy and a State Estimator for Dynamic and Robust Legged Locomotion
- Representation-Free Model Predictive Control for Dynamic Motions in Quadrupeds
- Cat-like Jumping and Landing of Legged Robots in Low-gravity Using Deep Reinforcement Learning
- Learning Free Gait Transition for Quadruped Robots via Phase-Guided Controller
- SoftGym: Benchmarking Deep Reinforcement Learning for Deformable Object Manipulation
- Iterative Reinforcement Learning Based Design of Dynamic Locomotion Skills for Cassie
- Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning
- Data-Driven Multi-Objective Controller Optimization for a Magnetically-Levitated Nanopositioning System
- Automated shapeshifting for function recovery in damaged robots
- PyRobot: An Open-source Robotics Framework for Research and Benchmarking
- BEHAVIOR: Benchmark for Everyday Household Activities in Virtual, Interactive, and Ecological Environments
- DeepRacer: Educational Autonomous Racing Platform for Experimentation with Sim2Real Reinforcement Learning
- Meta Reinforcement Learning for Optimal Design of Legged Robots
- Improved Learning of Robot Manipulation Tasks via Tactile Intrinsic Motivation
- How to pick the domain randomization parameters for sim-to-real transfer of reinforcement learning policies?
- Deep Value Model Predictive Control
- Asymmetric self-play for automatic goal discovery in robotic manipulation
- From Pixels to Legs: Hierarchical Learning of Quadruped Locomotion
- Modelling Generalized Forces with Reinforcement Learning for Sim-to-Real Transfer
- RMA: Rapid Motor Adaptation for Legged Robots
- Safe Multi-Agent Reinforcement Learning through Decentralized Multiple Control Barrier Functions
- Reinforcement Learning for Robust Parameterized Locomotion Control of Bipedal Robots
- Augmenting Differentiable Simulators with Neural Networks to Close the Sim2Real Gap
- Real-Time Model Calibration with Deep Reinforcement Learning
- Hierarchical Reinforcement Learning for Quadruped Locomotion
- Towards General and Autonomous Learning of Core Skills: A Case Study in Locomotion
- Traversing the Reality Gap via Simulator Tuning
- Context-Aware Safe Reinforcement Learning for Non-Stationary Environments
- Circus ANYmal: A Quadruped Learning Dexterous Manipulation with Its Limbs
- Zero-Shot Terrain Generalization for Visual Locomotion Policies
- Physically Embedded Planning Problems: New Challenges for Reinforcement Learning
- Informed Guided Rapidly-Exploring Random Trees*-Connect for Path Planning of Walking Robots
- A Nearly Optimal Chattering Reduction Method of Sliding Mode Control With an Application to a Two-wheeled Mobile Robot
- Minimizing Energy Consumption Leads to the Emergence of Gaits in Legged Robots
- Dynamics Randomization Revisited:A Case Study for Quadrupedal Locomotion
- Robust Quadrupedal Locomotion on Sloped Terrains: A Linear Policy Approach
- Comparing Semi-Parametric Model Learning Algorithms for Dynamic Model Estimation in Robotics
- Deep Reinforcement Learning in Fluid Mechanics: a promising method for both Active Flow Control and Shape Optimization
- Teach Biped Robots to Walk via Gait Principles and Reinforcement Learning with Adversarial Critics
- Combining Benefits from Trajectory Optimization and Deep Reinforcement Learning
- Learning Agile Locomotion via Adversarial Training
- Relevance-guided Unsupervised Discovery of Abilities with Quality-Diversity Algorithms
- Deep reinforcement learning for the control of conjugate heat transfer with application to workpiece cooling
- f-IRL: Inverse Reinforcement Learning via State Marginal Matching
- Machine-learning Based Extraction of the Short-Range Part of the Interaction in Non-contact Atomic Force Microscopy
- Gait Library Synthesis for Quadruped Robots via Augmented Random Search
- Learning to Locomote with Deep Neural-Network and CPG-based Control in a Soft Snake Robot
- Learning Generalizable Locomotion Skills with Hierarchical Reinforcement Learning
- A Learnable Safety Measure
- General Robot Dynamics Learning and Gen2Real
- Scalable sim-to-real transfer of soft robot designs
- Phoebe: Reuse-Aware Online Caching with Reinforcement Learning for Emerging Storage Models
- Deep Visual MPC-Policy Learning for Navigation
- Lyapunov-stable neural-network control
- Flying Through a Narrow Gap Using End-to-end Deep Reinforcement Learning Augmented with Curriculum Learning and Sim2Real
- Auditing Robot Learning for Safety and Compliance during Deployment
- Can Reinforcement Learning for Continuous Control Generalize Across Physics Engines?
- Learning Agile Locomotion Skills with a Mentor
- Protective Policy Transfer
- Deep Reinforcement Learning with Linear Quadratic Regulator Regions
- How does the structure embedded in learning policy affect learning quadruped locomotion?
- Meta-Reinforcement Learning for Adaptive Motor Control in Changing Robot Dynamics and Environments
- Improved Reinforcement Learning Coordinated Control of a Mobile Manipulator using Joint Clamping
- Two-stage training algorithm for AI robot soccer
- Out-of-the-box channel pruned networks
- Predicting Sim-to-Real Transfer with Probabilistic Dynamics Models
- Regret Bounds for Adaptive Nonlinear Control
- Adaptive Identification of Legged Robotic Kinematic Structure
- Task-Informed Fidelity Management for Speeding Up Robotics Simulation
- Quadruped Locomotion on Non-Rigid Terrain using Reinforcement Learning
- Contact Planning for the ANYmal Quadruped Robot using an Acyclic Reachability-Based Planner