Learning Quadrupedal Locomotion over Challenging Terrain

Abstract

Some of the most challenging environments on our planet are accessible to quadrupedal animals but remain out of reach for autonomous machines. Legged locomotion can dramatically expand the operational domains of robotics. However, conventional controllers for legged locomotion are based on elaborate state machines that explicitly trigger the execution of motion primitives and reflexes. These designs have escalated in complexity while falling short of the generality and robustness of animal locomotion. Here we present a radically robust controller for legged locomotion in challenging natural environments. We present a novel solution to incorporating proprioceptive feedback in locomotion control and demonstrate remarkable zero-shot generalization from simulation to natural environments. The controller is trained by reinforcement learning in simulation. It is based on a neural network that acts on a stream of proprioceptive signals. The trained controller has taken two generations of quadrupedal ANYmal robots to a variety of natural environments that are beyond the reach of prior published work in legged locomotion. The controller retains its robustness under conditions that have never been encountered during training: deformable terrain such as mud and snow, dynamic footholds such as rubble, and overground impediments such as thick vegetation and gushing water. The presented work opens new frontiers for robotics and indicates that radical robustness in natural environments can be achieved by training in much simpler domains.

References in corpus (5)

• Learning agile and dynamic motor skills for legged robots
• Sim-to-Real Transfer of Robotic Control with Dynamics Randomization
• Learning Symmetric and Low-energy Locomotion
• Learning Ground Traversability from Simulations
• Heuristic Planning for Rough Terrain Locomotion in Presence of External Disturbances and Variable Perception Quality

Cited by in corpus (95)

• Learning robust perceptive locomotion for quadrupedal robots in the wild
• Seeing biodiversity: perspectives in machine learning for wildlife conservation
• How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learned
• Neural-Fly Enables Rapid Learning for Agile Flight in Strong Winds
• Reaching the Limit in Autonomous Racing: Optimal Control versus Reinforcement Learning
• Concurrent Training of a Control Policy and a State Estimator for Dynamic and Robust Legged Locomotion
• Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learning
• RLOC: Terrain-Aware Legged Locomotion using Reinforcement Learning and Optimal Control
• Deep Reinforcement Learning with Shallow Controllers: An Experimental Application to PID Tuning
• Learning Robust Autonomous Navigation and Locomotion for Wheeled-Legged Robots
• DTC: Deep Tracking Control
• Scientific Exploration of Challenging Planetary Analog Environments with a Team of Legged Robots
• Super-Human Performance in Gran Turismo Sport Using Deep Reinforcement Learning
• Visual Dexterity: In-Hand Reorientation of Novel and Complex Object Shapes
• Learning Free Gait Transition for Quadruped Robots via Phase-Guided Controller
• Versatile Multi-Contact Planning and Control for Legged Loco-Manipulation
• Measuring and modeling the motor system with machine learning
• Robust High-speed Running for Quadruped Robots via Deep Reinforcement Learning
• Learning Minimum-Time Flight in Cluttered Environments
• Lifelike Agility and Play in Quadrupedal Robots using Reinforcement Learning and Generative Pre-trained Models
• Robot Learning of Mobile Manipulation with Reachability Behavior Priors
• RL + Model-based Control: Using On-demand Optimal Control to Learn Versatile Legged Locomotion
• Meta-Reinforcement Learning for the Tuning of PI Controllers: An Offline Approach
• Controlling the Solo12 Quadruped Robot with Deep Reinforcement Learning
• ViTAL: Vision-Based Terrain-Aware Locomotion for Legged Robots
• Traversability Analysis for Autonomous Driving in Complex Environment: A LiDAR-based Terrain Modeling Approach
• Multi-legged matter transport: a framework for locomotion on noisy landscapes
• Learning-based Design and Control for Quadrupedal Robots with Parallel-Elastic Actuators
• Learning Bipedal Walking for Humanoids with Current Feedback
• Optimizing Bipedal Locomotion for The 100m Dash With Comparison to Human Running
• Grasp Multiple Objects with One Hand
• An Efficient Locally Reactive Controller for Safe Navigation in Visual Teach and Repeat Missions
• Meta Reinforcement Learning for Optimal Design of Legged Robots
• Managing power grids through topology actions: A comparative study between advanced rule-based and reinforcement learning agents
• Enhancing Efficiency and Propulsion in Bio-mimetic Robotic Fish through End-to-End Deep Reinforcement Learning
• Torque-based Deep Reinforcement Learning for Task-and-Robot Agnostic Learning on Bipedal Robots Using Sim-to-Real Transfer
• Navigating by Touch: Haptic Monte Carlo Localization via Geometric Sensing and Terrain Classification
• Imitating and Finetuning Model Predictive Control for Robust and Symmetric Quadrupedal Locomotion
• Learning coordinated badminton skills for legged manipulators
• The need for and feasibility of alternative ground robots to traverse sandy and rocky extraterrestrial terrain
• Reinforcement Learning for Photonic Component Design
• Learning Whole-Body Loco-Manipulation for Omni-Directional Task Space Pose Tracking with a Wheeled-Quadrupedal-Manipulator
• Leveraging Symmetry in RL-based Legged Locomotion Control
• UnifiedGesture: A Unified Gesture Synthesis Model for Multiple Skeletons
• A Legged Soft Robot Platform for Dynamic Locomotion
• A Learning-based Quadcopter Controller with Extreme Adaptation
• Imperative Learning: A Self-supervised Neuro-Symbolic Learning Framework for Robot Autonomy
• Robust Humanoid Walking on Compliant and Uneven Terrain with Deep Reinforcement Learning
• Vision-based Navigation of Unmanned Aerial Vehicles in Orchards: An Imitation Learning Approach
• Symbolic Equation Solving via Reinforcement Learning
• Learn to Teach: Sample-Efficient Privileged Learning for Humanoid Locomotion over Diverse Terrains
• Learning to Boost the Performance of Stable Nonlinear Systems
• Morphological Symmetries in Robotics
• MEVIUS: A Quadruped Robot Easily Constructed through E-Commerce with Sheet Metal Welding and Machining
• HUGO -- Highlighting Unseen Grid Options: Combining Deep Reinforcement Learning with a Heuristic Target Topology Approach
• Learning Quadrupedal Locomotion for a Heavy Hydraulic Robot Using an Actuator Model
• Online Friction Coefficient Identification for Legged Robots on Slippery Terrain Using Smoothed Contact Gradients
• An Adaptable Approach to Learn Realistic Legged Locomotion without Examples
• Learning-based Adaptive Control of Quadruped Robots for Active Stabilization on Moving Platforms
• An Invitation to Deep Reinforcement Learning
• Environmentally Adaptive Control Including Variance Minimization Using Stochastic Predictive Network with Parametric Bias: Application to Mobile Robots
• Practice Makes Perfect: an iterative approach to achieve precise tracking for legged robots
• Noise Analysis and Modeling of the PMD Flexx2 Depth Camera for Robotic Applications
• TAR: Teacher-Aligned Representations via Contrastive Learning for Quadrupedal Locomotion
• Machine Learning for Quantum-Enhanced Gravitational-Wave Observatories
• Visual-Policy Learning through Multi-Camera View to Single-Camera View Knowledge Distillation for Robot Manipulation Tasks
• Deep Predictive Model Learning with Parametric Bias: Handling Modeling Difficulties and Temporal Model Changes
• Swift progress for robots over complex terrain
• Efficient Learning of Control Policies for Robust Quadruped Bounding using Pretrained Neural Networks
• Mechanical Intelligence-Aware Curriculum Reinforcement Learning for Humanoids with Parallel Actuation
• Stretchable Electrohydraulic Artificial Muscle for Full Motion Ranges in Musculoskeletal Antagonistic Joints
• Adaptive Wiping: Adaptive contact-rich manipulation through few-shot imitation learning with Force-Torque feedback and pre-trained object representations
• PPF: Pre-training and Preservative Fine-tuning of Humanoid Locomotion via Model-Assumption-based Regularization
• End-to-End Crop Row Navigation via LiDAR-Based Deep Reinforcement Learning
• DreamWaQ++: Obstacle-Aware Quadrupedal Locomotion With Resilient Multi-Modal Reinforcement Learning
• Legged Robot State Estimation Using Invariant Neural-Augmented Kalman Filter with a Neural Compensator
• Simulation-Aided Policy Tuning for Black-Box Robot Learning
• Enhancing Navigation Efficiency of Quadruped Robots via Leveraging Personal Transportation Platforms
• Body Design and Gait Generation of Chair-Type Asymmetrical Tripedal Low-rigidity Robot
• Learning Fast and Precise Pixel-to-Torque Control
• Vision-Proprioception Fusion with Mamba2 in End-to-End Reinforcement Learning for Motion Control
• Probabilistic approach to feedback control enhances multi-legged locomotion on rugged landscapes
• Growable and Interpretable Neural Control with Online Continual Learning for Autonomous Lifelong Locomotion Learning Machines
• Learning to Predict Contact Force Distributions from Vision Leveraging Object Geometry Priors
• Energy-Efficient Omnidirectional Locomotion for Wheeled Quadrupeds via Predictive Energy-Aware Nominal Gait Selection
• SHIELD: Safety on Humanoids via CBFs In Expectation on Learned Dynamics
• Situationally-Aware Dynamics Learning
• LocoMamba: Vision-Driven Locomotion via End-to-End Deep Reinforcement Learning with Mamba
• Robust Reinforcement Learning-Based Locomotion for Resource-Constrained Quadrupeds with Exteroceptive Sensing
• Learning Terrain-Specialized Policies for Adaptive Locomotion in Challenging Environments
• Seq-DeepIPC: Sequential Sensing for End-to-End Control in Legged Robot Navigation
• Constrained Performance Boosting Control for Nonlinear Systems
• BinWalker: Development and Field Evaluation of a Quadruped Manipulator Platform for Sustainable Litter Collection
• Agile perceptive multi-skill locomotion for quadrupedal robots in the wild
• Advances, challenges, and opportunities for legged robots