Lifelike Agility and Play in Quadrupedal Robots using Reinforcement Learning and Generative Pre-trained Models

Abstract

Knowledge from animals and humans inspires robotic innovations. Numerous efforts have been made to achieve agile locomotion in quadrupedal robots through classical controllers or reinforcement learning approaches. These methods usually rely on physical models or handcrafted rewards to accurately describe the specific system, rather than on a generalized understanding like animals do. Here we propose a hierarchical framework to construct primitive-, environmental- and strategic-level knowledge that are all pre-trainable, reusable and enrichable for legged robots. The primitive module summarizes knowledge from animal motion data, where, inspired by large pre-trained models in language and image understanding, we introduce deep generative models to produce motor control signals stimulating legged robots to act like real animals. Then, we shape various traversing capabilities at a higher level to align with the environment by reusing the primitive module. Finally, a strategic module is trained focusing on complex downstream tasks by reusing the knowledge from previous levels. We apply the trained hierarchical controllers to the MAX robot, a quadrupedal robot developed in-house, to mimic animals, traverse complex obstacles and play in a designed challenging multi-agent chase tag game, where lifelike agility and strategy emerge in the robots.

Published in Nature Machine Intelligence, Vol. 7, 2024

References in corpus (9)

• Learning agile and dynamic motor skills for legged robots
• Learning robust perceptive locomotion for quadrupedal robots in the wild
• Character Controllers Using Motion VAEs
• Multi-expert learning of adaptive legged locomotion
• Imitate and Repurpose: Learning Reusable Robot Movement Skills From Human and Animal Behaviors
• Behavior Priors for Efficient Reinforcement Learning
• Barkour: Benchmarking Animal-level Agility with Quadruped Robots
• TLeague: A Framework for Competitive Self-Play based Distributed Multi-Agent Reinforcement Learning
• CAJun: Continuous Adaptive Jumping using a Learned Centroidal Controller

Cited by in corpus (6)

• HumanMimic: Learning Natural Locomotion and Transitions for Humanoid Robot via Wasserstein Adversarial Imitation
• Topology-Aware and Highly Generalizable Deep Reinforcement Learning for Efficient Retrieval in Multi-Deep Storage Systems
• End-to-End Crop Row Navigation via LiDAR-Based Deep Reinforcement Learning
• Quadrupped-Legged Robot Movement Plan Generation using Large Language Model
• Agile perceptive multi-skill locomotion for quadrupedal robots in the wild
• Learning Terrain-Specialized Policies for Adaptive Locomotion in Challenging Environments