Learning Vision-Guided Quadrupedal Locomotion End-to-End with Cross-Modal Transformers

Abstract

We propose to address quadrupedal locomotion tasks using Reinforcement Learning (RL) with a Transformer-based model that learns to combine proprioceptive information and high-dimensional depth sensor inputs. While learning-based locomotion has made great advances using RL, most methods still rely on domain randomization for training blind agents that generalize to challenging terrains. Our key insight is that proprioceptive states only offer contact measurements for immediate reaction, whereas an agent equipped with visual sensory observations can learn to proactively maneuver environments with obstacles and uneven terrain by anticipating changes in the environment many steps ahead. In this paper, we introduce LocoTransformer, an end-to-end RL method that leverages both proprioceptive states and visual observations for locomotion control. We evaluate our method in challenging simulated environments with different obstacles and uneven terrain. We transfer our learned policy from simulation to a real robot by running it indoors and in the wild with unseen obstacles and terrain. Our method not only significantly improves over baselines, but also achieves far better generalization performance, especially when transferred to the real robot. Our project page with videos is at https://rchalyang.github.io/LocoTransformer/ .

arXiv commentOur project page with videos is at https://RchalYang.github.io/LocoTransformer

References in corpus (27)

• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• Language Models are Few-Shot Learners
• High-Dimensional Continuous Control Using Generalized Advantage Estimation
• Learning agile and dynamic motor skills for legged robots
• VisualBERT: A Simple and Performant Baseline for Vision and Language
• VL-BERT: Pre-training of Generic Visual-Linguistic Representations
• Emergence of Locomotion Behaviours in Rich Environments
• IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures
• Generating Long Sequences with Sparse Transformers
• More Than a Feeling: Learning to Grasp and Regrasp using Vision and Touch
• Learning to Navigate in Complex Environments
• VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Text
• Reinforcement Learning with Unsupervised Auxiliary Tasks
• Image Transformer
• LXMERT: Learning Cross-Modality Encoder Representations from Transformers
• Dream to Control: Learning Behaviors by Latent Imagination
• Language as an Abstraction for Hierarchical Deep Reinforcement Learning
• Decoupling Representation Learning from Reinforcement Learning
• Data-Efficient Reinforcement Learning with Self-Predictive Representations
• Mid-Level Visual Representations Improve Generalization and Sample Efficiency for Learning Visuomotor Policies
• Policies Modulating Trajectory Generators
• Visual Semantic Navigation using Scene Priors
• Learning a Contact-Adaptive Controller for Robust, Efficient Legged Locomotion
• From Pixels to Legs: Hierarchical Learning of Quadruped Locomotion
• RMA: Rapid Motor Adaptation for Legged Robots
• Decoupling the Role of Data, Attention, and Losses in Multimodal Transformers
• GLiDE: Generalizable Quadrupedal Locomotion in Diverse Environments with a Centroidal Model

Cited by in corpus (4)

• Robust High-speed Running for Quadruped Robots via Deep Reinforcement Learning
• Learning to Generate Pointing Gestures in Situated Embodied Conversational Agents
• Multimodal Information Bottleneck for Deep Reinforcement Learning with Multiple Sensors
• Vision-Guided Quadrupedal Locomotion in the Wild with Multi-Modal Delay Randomization