Energy-based Legged Robots Terrain Traversability Modeling via Deep Inverse Reinforcement Learning

Abstract

This work reports on developing a deep inverse reinforcement learning method for legged robots terrain traversability modeling that incorporates both exteroceptive and proprioceptive sensory data. Existing works use robot-agnostic exteroceptive environmental features or handcrafted kinematic features; instead, we propose to also learn robot-specific inertial features from proprioceptive sensory data for reward approximation in a single deep neural network. Incorporating the inertial features can improve the model fidelity and provide a reward that depends on the robot's state during deployment. We train the reward network using the Maximum Entropy Deep Inverse Reinforcement Learning (MEDIRL) algorithm and propose simultaneously minimizing a trajectory ranking loss to deal with the suboptimality of legged robot demonstrations. The demonstrated trajectories are ranked by locomotion energy consumption, in order to learn an energy-aware reward function and a more energy-efficient policy than demonstration. We evaluate our method using a dataset collected by an MIT Mini-Cheetah robot and a Mini-Cheetah simulator. The code is publicly available at https://github.com/ganlumomo/minicheetah-traversability-irl.

References in corpus (13)

• ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Cameras
• Road Extraction by Deep Residual U-Net
• An Algorithmic Perspective on Imitation Learning
• Maximum Entropy Deep Inverse Reinforcement Learning
• Trajectory Forecasts in Unknown Environments Conditioned on Grid-Based Plans
• Bayesian Spatial Kernel Smoothing for Scalable Dense Semantic Mapping
• Sampling-based Incremental Information Gathering with Applications to Robotic Exploration and Environmental Monitoring
• Toward Safety-Aware Informative Motion Planning for Legged Robots
• Navigating by Touch: Haptic Monte Carlo Localization via Geometric Sensing and Terrain Classification
• Multitask Learning for Scalable and Dense Multilayer Bayesian Map Inference
• Hybrid Imitative Planning with Geometric and Predictive Costs in Off-road Environments
• Efficient Anytime CLF Reactive Planning System for a Bipedal Robot on Undulating Terrain
• Inverse reinforcement learning for autonomous navigation via differentiable semantic mapping and planning

Cited by in corpus (3)

• Multitask Learning for Scalable and Dense Multilayer Bayesian Map Inference
• Traversability-aware Adaptive Optimization for Path Planning and Control in Mountainous Terrain
• STEP: Stochastic Traversability Evaluation and Planning for Risk-Aware Off-road Navigation; Results from the DARPA Subterranean Challenge