Learning robust perceptive locomotion for quadrupedal robots in the wild

Abstract

Legged robots that can operate autonomously in remote and hazardous environments will greatly increase opportunities for exploration into under-explored areas. Exteroceptive perception is crucial for fast and energy-efficient locomotion: perceiving the terrain before making contact with it enables planning and adaptation of the gait ahead of time to maintain speed and stability. However, utilizing exteroceptive perception robustly for locomotion has remained a grand challenge in robotics. Snow, vegetation, and water visually appear as obstacles on which the robot cannot step~-- or are missing altogether due to high reflectance. Additionally, depth perception can degrade due to difficult lighting, dust, fog, reflective or transparent surfaces, sensor occlusion, and more. For this reason, the most robust and general solutions to legged locomotion to date rely solely on proprioception. This severely limits locomotion speed, because the robot has to physically feel out the terrain before adapting its gait accordingly. Here we present a robust and general solution to integrating exteroceptive and proprioceptive perception for legged locomotion. We leverage an attention-based recurrent encoder that integrates proprioceptive and exteroceptive input. The encoder is trained end-to-end and learns to seamlessly combine the different perception modalities without resorting to heuristics. The result is a legged locomotion controller with high robustness and speed. The controller was tested in a variety of challenging natural and urban environments over multiple seasons and completed an hour-long hike in the Alps in the time recommended for human hikers.

References in corpus (6)

• Learning agile and dynamic motor skills for legged robots
• Learning Quadrupedal Locomotion over Challenging Terrain
• DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills
• Multi-expert learning of adaptive legged locomotion
• Fast and Continuous Foothold Adaptation for Dynamic Locomotion through CNNs
• Heuristic Planning for Rough Terrain Locomotion in Presence of External Disturbances and Variable Perception Quality

Cited by in corpus (73)

• Reaching the Limit in Autonomous Racing: Optimal Control versus Reinforcement Learning
• Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learning
• RLOC: Terrain-Aware Legged Locomotion using Reinforcement Learning and Optimal Control
• Learning Robust Autonomous Navigation and Locomotion for Wheeled-Legged Robots
• DTC: Deep Tracking Control
• Scientific Exploration of Challenging Planetary Analog Environments with a Team of Legged Robots
• Versatile Multi-Contact Planning and Control for Legged Loco-Manipulation
• WayFAST: Navigation with Predictive Traversability in the Field
• Robust High-speed Running for Quadruped Robots via Deep Reinforcement Learning
• Locomotion Policy Guided Traversability Learning using Volumetric Representations of Complex Environments
• Lifelike Agility and Play in Quadrupedal Robots using Reinforcement Learning and Generative Pre-trained Models
• RL + Model-based Control: Using On-demand Optimal Control to Learn Versatile Legged Locomotion
• A Tutorial on Meta-Reinforcement Learning
• Back to Newton's Laws: Learning Vision-based Agile Flight via Differentiable Physics
• Controlling the Solo12 Quadruped Robot with Deep Reinforcement Learning
• ViTAL: Vision-Based Terrain-Aware Locomotion for Legged Robots
• Learning-based Design and Control for Quadrupedal Robots with Parallel-Elastic Actuators
• Testing of Deep Reinforcement Learning Agents with Surrogate Models
• Grasp Multiple Objects with One Hand
• Meta Reinforcement Learning for Optimal Design of Legged Robots
• Reconstructing occluded Elevation Information in Terrain Maps with Self-supervised Learning
• Benchmarking Potential Based Rewards for Learning Humanoid Locomotion
• Imitating and Finetuning Model Predictive Control for Robust and Symmetric Quadrupedal Locomotion
• Learning coordinated badminton skills for legged manipulators
• Reactive Stepping for Humanoid Robots using Reinforcement Learning: Application to Standing Push Recovery on the Exoskeleton Atalante
• Learning Terrain-Aware Kinodynamic Model for Autonomous Off-Road Rally Driving With Model Predictive Path Integral Control
• DeepCPG Policies for Robot Locomotion
• Offline Supervised Learning V.S. Online Direct Policy Optimization: A Comparative Study and A Unified Training Paradigm for Neural Network-Based Optimal Feedback Control
• Hierarchical Quality-Diversity for Online Damage Recovery
• Robust Humanoid Walking on Compliant and Uneven Terrain with Deep Reinforcement Learning
• Planar Modeling and Sim-to-Real of a Tethered Multimaterial Soft Swimmer Driven by Peano-HASELs
• FR-Net: Learning Robust Quadrupedal Fall Recovery on Challenging Terrains through Mass-Contact Prediction
• Learn to Teach: Sample-Efficient Privileged Learning for Humanoid Locomotion over Diverse Terrains
• MEVIUS: A Quadruped Robot Easily Constructed through E-Commerce with Sheet Metal Welding and Machining
• RESPLE: Recursive Spline Estimation for LiDAR-Based Odometry
• Informed Guided Rapidly-Exploring Random Trees*-Connect for Path Planning of Walking Robots
• RENet: Fault-Tolerant Motion Control for Quadruped Robots via Redundant Estimator Networks under Visual Collapse
• Learning-based Adaptive Control of Quadruped Robots for Active Stabilization on Moving Platforms
• Nature's All-in-One: Multitasking Robots Inspired by Dung Beetles
• Non-Gaited Legged Locomotion with Monte-Carlo Tree Search and Supervised Learning
• Agile and Safe Trajectory Planning for Quadruped Navigation with Motion Anisotropy Awareness
• An Invitation to Deep Reinforcement Learning
• TAR: Teacher-Aligned Representations via Contrastive Learning for Quadrupedal Locomotion
• SafeSteps: Learning Safer Footstep Planning Policies for Legged Robots via Model-Based Priors
• Learning of Balance Controller Considering Changes in Body State for Musculoskeletal Humanoids
• Visual-Policy Learning through Multi-Camera View to Single-Camera View Knowledge Distillation for Robot Manipulation Tasks
• Noise Analysis and Modeling of the PMD Flexx2 Depth Camera for Robotic Applications
• A Pin-Array Structure for Gripping and Shape Recognition of Convex and Concave Terrain Profiles
• Kernel-based diffusion approximated Markov decision processes for autonomous navigation and control on unstructured terrains
• Neuromorphic force-control in an industrial task: validating energy and latency benefits
• Stretchable Electrohydraulic Artificial Muscle for Full Motion Ranges in Musculoskeletal Antagonistic Joints
• DreamWaQ++: Obstacle-Aware Quadrupedal Locomotion With Resilient Multi-Modal Reinforcement Learning
• End-to-End Crop Row Navigation via LiDAR-Based Deep Reinforcement Learning
• PPF: Pre-training and Preservative Fine-tuning of Humanoid Locomotion via Model-Assumption-based Regularization
• Legged Robot State Estimation Using Invariant Neural-Augmented Kalman Filter with a Neural Compensator
• Hybrid Dynamics Modeling and Trajectory Planning for a Cable-Trailer System with a Quadruped Robot
• Real-time Digital Double Framework to Predict Collapsible Terrains for Legged Robots
• Enhancing Navigation Efficiency of Quadruped Robots via Leveraging Personal Transportation Platforms
• Autonomous Legged Mobile Manipulation for Lunar Surface Operations via Constrained Reinforcement Learning
• Benchmarking Population-Based Reinforcement Learning across Robotic Tasks with GPU-Accelerated Simulation
• Optimal Gait Control for a Tendon-driven Soft Quadruped Robot by Model-based Reinforcement Learning
• Probabilistic approach to feedback control enhances multi-legged locomotion on rugged landscapes
• Continuous Jumping of a Parallel Wire-Driven Monopedal Robot RAMIEL Using Reinforcement Learning
• Robust Reinforcement Learning-Based Locomotion for Resource-Constrained Quadrupeds with Exteroceptive Sensing
• SHIELD: Safety on Humanoids via CBFs In Expectation on Learned Dynamics
• EquiMus: Energy-Equivalent Dynamic Modeling and Simulation of Musculoskeletal Robots Driven by Linear Elastic Actuators
• Learning Terrain-Specialized Policies for Adaptive Locomotion in Challenging Environments
• Generating Diverse Challenging Terrains for Legged Robots Using Quality-Diversity Algorithm
• Agile perceptive multi-skill locomotion for quadrupedal robots in the wild
• Advances, challenges, and opportunities for legged robots
• Hamilton-Jacobi Reachability for Hybrid Systems: Unified Goal-Driven Control with Safety Guarantees
• Torque-Minimizing Control Allocation for Overactuated Quadrupedal Locomotion
• Seq-DeepIPC: Sequential Sensing for End-to-End Control in Legged Robot Navigation