Learning natural locomotion behaviors for humanoid robots using human knowledge

Abstract

This paper presents a new learning framework that leverages the knowledge from imitation learning, deep reinforcement learning, and control theories to achieve human-style locomotion that is natural, dynamic, and robust for humanoids. We proposed novel approaches to introduce human bias, i.e. motion capture data and a special Multi-Expert network structure. We used the Multi-Expert network structure to smoothly blend behavioral features, and used the augmented reward design for the task and imitation rewards. Our reward design is composable, tunable, and explainable by using fundamental concepts from conventional humanoid control. We rigorously validated and benchmarked the learning framework which consistently produced robust locomotion behaviors in various test scenarios. Further, we demonstrated the capability of learning robust and versatile policies in the presence of disturbances, such as terrain irregularities and external pushes.

arXiv commentuniversity policy

References in corpus (4)

• Learning agile and dynamic motor skills for legged robots
• Learning Locomotion Skills Using DeepRL: Does the Choice of Action Space Matter?
• Learning Whole-body Motor Skills for Humanoids
• Emergence of Human-comparable Balancing Behaviors by Deep Reinforcement Learning

Cited by in corpus (6)

• Multi-expert learning of adaptive legged locomotion
• Contact-Implicit Trajectory Optimization using an Analytically Solvable Contact Model for Locomotion on Variable Ground
• On the Emergence of Whole-body Strategies from Humanoid Robot Push-recovery Learning
• Reactive Stepping for Humanoid Robots using Reinforcement Learning: Application to Standing Push Recovery on the Exoskeleton Atalante
• Deep Reinforcement Learning for Bipedal Locomotion: A Brief Survey
• Robust Humanoid Walking on Compliant and Uneven Terrain with Deep Reinforcement Learning