Learning Bipedal Walking for Humanoids with Current Feedback

Abstract

Recent advances in deep reinforcement learning (RL) based techniques combined with training in simulation have offered a new approach to developing robust controllers for legged robots. However, the application of such approaches to real hardware has largely been limited to quadrupedal robots with direct-drive actuators and light-weight bipedal robots with low gear-ratio transmission systems. Application to real, life-sized humanoid robots has been less common arguably due to a large sim2real gap. In this paper, we present an approach for effectively overcoming the sim2real gap issue for humanoid robots arising from inaccurate torque-tracking at the actuator level. Our key idea is to utilize the current feedback from the actuators on the real robot, after training the policy in a simulation environment artificially degraded with poor torque-tracking. Our approach successfully trains a unified, end-to-end policy in simulation that can be deployed on a real HRP-5P humanoid robot to achieve bipedal locomotion. Through ablations, we also show that a feedforward policy architecture combined with targeted dynamics randomization is sufficient for zero-shot sim2real success, thus eliminating the need for computationally expensive, memory-based network architectures. Finally, we validate the robustness of the proposed RL policy by comparing its performance against a conventional model-based controller for walking on uneven terrain with the real robot.

References in corpus (6)

• Learning agile and dynamic motor skills for legged robots
• Learning Quadrupedal Locomotion over Challenging Terrain
• Multi-expert learning of adaptive legged locomotion
• Centroidal Trajectory Generation and Stabilization based on Preview Control for Humanoid Multi-contact Motion
• Torque-based Deep Reinforcement Learning for Task-and-Robot Agnostic Learning on Bipedal Robots Using Sim-to-Real Transfer
• Learning Whole-body Motor Skills for Humanoids

Cited by in corpus (5)

• HumanMimic: Learning Natural Locomotion and Transitions for Humanoid Robot via Wasserstein Adversarial Imitation
• Physics-Informed Neural Networks with Unscented Kalman Filter for Sensorless Joint Torque Estimation in Humanoid Robots
• PDP: Physics-Based Character Animation via Diffusion Policy
• Deep Reinforcement Learning for Bipedal Locomotion: A Brief Survey
• Robust Humanoid Walking on Compliant and Uneven Terrain with Deep Reinforcement Learning