RLOC: Terrain-Aware Legged Locomotion using Reinforcement Learning and Optimal Control

Abstract

We present a unified model-based and data-driven approach for quadrupedal planning and control to achieve dynamic locomotion over uneven terrain. We utilize on-board proprioceptive and exteroceptive feedback to map sensory information and desired base velocity commands into footstep plans using a reinforcement learning (RL) policy. This RL policy is trained in simulation over a wide range of procedurally generated terrains. When ran online, the system tracks the generated footstep plans using a model-based motion controller. We evaluate the robustness of our method over a wide variety of complex terrains. It exhibits behaviors which prioritize stability over aggressive locomotion. Additionally, we introduce two ancillary RL policies for corrective whole-body motion tracking and recovery control. These policies account for changes in physical parameters and external perturbations. We train and evaluate our framework on a complex quadrupedal system, ANYmal version B, and demonstrate transferability to a larger and heavier robot, ANYmal C, without requiring retraining.

26 pages, 19 figures, 16 tables, 2 algorithms, accepted for publication to IEEE T-RO

References in corpus (4)

• Learning agile and dynamic motor skills for legged robots
• Learning Quadrupedal Locomotion over Challenging Terrain
• Learning robust perceptive locomotion for quadrupedal robots in the wild
• Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning

Cited by in corpus (14)

• VILENS: Visual, Inertial, Lidar, and Leg Odometry for All-Terrain Legged Robots
• Learning Robust Autonomous Navigation and Locomotion for Wheeled-Legged Robots
• DTC: Deep Tracking Control
• TAMOLS: Terrain-Aware Motion Optimization for Legged Systems
• Learning Free Gait Transition for Quadruped Robots via Phase-Guided Controller
• Torque-based Deep Reinforcement Learning for Task-and-Robot Agnostic Learning on Bipedal Robots Using Sim-to-Real Transfer
• Deep Reinforcement Learning for Bipedal Locomotion: A Brief Survey
• Learning to Brachiate via Simplified Model Imitation
• GLiDE: Generalizable Quadrupedal Locomotion in Diverse Environments with a Centroidal Model
• Learning-based Adaptive Control of Quadruped Robots for Active Stabilization on Moving Platforms
• SafeSteps: Learning Safer Footstep Planning Policies for Legged Robots via Model-Based Priors
• DreamWaQ++: Obstacle-Aware Quadrupedal Locomotion With Resilient Multi-Modal Reinforcement Learning
• Optimal Gait Control for a Tendon-driven Soft Quadruped Robot by Model-based Reinforcement Learning
• Quadrupedal Footstep Planning using Learned Motion Models of a Black-Box Controller