GLiDE: Generalizable Quadrupedal Locomotion in Diverse Environments with a Centroidal Model

Abstract

Model-free reinforcement learning (RL) for legged locomotion commonly relies on a physics simulator that can accurately predict the behaviors of every degree of freedom of the robot. In contrast, approximate reduced-order models are commonly used for many model predictive control strategies. In this work we abandon the conventional use of high-fidelity dynamics models in RL and we instead seek to understand what can be achieved when using RL with a much simpler centroidal model when applied to quadrupedal locomotion. We show that RL-based control of the accelerations of a centroidal model is surprisingly effective, when combined with a quadratic program to realize the commanded actions via ground contact forces. It allows for a simple reward structure, reduced computational costs, and robust sim-to-real transfer. We show the generality of the method by demonstrating flat-terrain gaits, stepping-stone locomotion, two-legged in-place balance, balance beam locomotion, and direct sim-to-real transfer.

arXiv commentvideo: https://youtu.be/m4scMY8LcmQ

References in corpus (12)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Learning agile and dynamic motor skills for legged robots
• Motion Planning for Quadrupedal Locomotion: Coupled Planning, Terrain Mapping and Whole-Body Control
• RLOC: Terrain-Aware Legged Locomotion using Reinforcement Learning and Optimal Control
• Sim-to-Real: Learning Agile Locomotion For Quadruped Robots
• Preparing for the Unknown: Learning a Universal Policy with Online System Identification
• Learning a Contact-Adaptive Controller for Robust, Efficient Legged Locomotion
• Angular Momentum about the Contact Point for Control of Bipedal Locomotion: Validation in a LIP-based Controller
• From Pixels to Legs: Hierarchical Learning of Quadruped Locomotion
• Robust Quadrupedal Locomotion on Sloped Terrains: A Linear Policy Approach
• Dynamics Randomization Revisited:A Case Study for Quadrupedal Locomotion
• Planning in Learned Latent Action Spaces for Generalizable Legged Locomotion

Cited by in corpus (3)

• Learning Vision-Guided Quadrupedal Locomotion End-to-End with Cross-Modal Transformers
• An Adaptable Approach to Learn Realistic Legged Locomotion without Examples
• Solving Challenging Control Problems Using Two-Staged Deep Reinforcement Learning