Concurrent Training of a Control Policy and a State Estimator for Dynamic and Robust Legged Locomotion

Abstract

In this paper, we propose a locomotion training framework where a control policy and a state estimator are trained concurrently. The framework consists of a policy network which outputs the desired joint positions and a state estimation network which outputs estimates of the robot's states such as the base linear velocity, foot height, and contact probability. We exploit a fast simulation environment to train the networks and the trained networks are transferred to the real robot. The trained policy and state estimator are capable of traversing diverse terrains such as a hill, slippery plate, and bumpy road. We also demonstrate that the learned policy can run at up to 3.75 m/s on normal flat ground and 3.54 m/s on a slippery plate with the coefficient of friction of 0.22.

arXiv commentAccepted for IEEE Robotics and Automation Letters (RA-L) and ICRA 2022

References in corpus (7)

• Learning agile and dynamic motor skills for legged robots
• Learning Quadrupedal Locomotion over Challenging Terrain
• Sim-to-Real Transfer of Robotic Control with Dynamics Randomization
• Highly Dynamic Quadruped Locomotion via Whole-Body Impulse Control and Model Predictive Control
• Multi-expert learning of adaptive legged locomotion
• Representation-Free Model Predictive Control for Dynamic Motions in Quadrupeds
• Learning to Locomote: Understanding How Environment Design Matters for Deep Reinforcement Learning

Cited by in corpus (16)

• Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learning
• Learning Robust Autonomous Navigation and Locomotion for Wheeled-Legged Robots
• Versatile Multi-Contact Planning and Control for Legged Loco-Manipulation
• Robust High-speed Running for Quadruped Robots via Deep Reinforcement Learning
• Controlling the Solo12 Quadruped Robot with Deep Reinforcement Learning
• Contact-Implicit Model Predictive Control: Controlling Diverse Quadruped Motions Without Pre-Planned Contact Modes or Trajectories
• Imitating and Finetuning Model Predictive Control for Robust and Symmetric Quadrupedal Locomotion
• Learning coordinated badminton skills for legged manipulators
• Learning Quadrupedal Locomotion for a Heavy Hydraulic Robot Using an Actuator Model
• Learning-based Adaptive Control of Quadruped Robots for Active Stabilization on Moving Platforms
• Efficient Reinforcement Learning for Jumping Monopods
• DreamWaQ++: Obstacle-Aware Quadrupedal Locomotion With Resilient Multi-Modal Reinforcement Learning
• Legged Robot State Estimation Using Invariant Neural-Augmented Kalman Filter with a Neural Compensator
• Enhancing Navigation Efficiency of Quadruped Robots via Leveraging Personal Transportation Platforms
• Agile perceptive multi-skill locomotion for quadrupedal robots in the wild
• Robust Reinforcement Learning-Based Locomotion for Resource-Constrained Quadrupeds with Exteroceptive Sensing