DTC: Deep Tracking Control

Abstract

Legged locomotion is a complex control problem that requires both accuracy and robustness to cope with real-world challenges. Legged systems have traditionally been controlled using trajectory optimization with inverse dynamics. Such hierarchical model-based methods are appealing due to intuitive cost function tuning, accurate planning, generalization, and most importantly, the insightful understanding gained from more than one decade of extensive research. However, model mismatch and violation of assumptions are common sources of faulty operation. Simulation-based reinforcement learning, on the other hand, results in locomotion policies with unprecedented robustness and recovery skills. Yet, all learning algorithms struggle with sparse rewards emerging from environments where valid footholds are rare, such as gaps or stepping stones. In this work, we propose a hybrid control architecture that combines the advantages of both worlds to simultaneously achieve greater robustness, foot-placement accuracy, and terrain generalization. Our approach utilizes a model-based planner to roll out a reference motion during training. A deep neural network policy is trained in simulation, aiming to track the optimized footholds. We evaluate the accuracy of our locomotion pipeline on sparse terrains, where pure data-driven methods are prone to fail. Furthermore, we demonstrate superior robustness in the presence of slippery or deformable ground when compared to model-based counterparts. Finally, we show that our proposed tracking controller generalizes across different trajectory optimization methods not seen during training. In conclusion, our work unites the predictive capabilities and optimality guarantees of online planning with the inherent robustness attributed to offline learning.

References in corpus (8)

• Learning agile and dynamic motor skills for legged robots
• Learning Quadrupedal Locomotion over Challenging Terrain
• DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills
• Learning robust perceptive locomotion for quadrupedal robots in the wild
• AMP: Adversarial Motion Priors for Stylized Physics-Based Character Control
• Motion Planning for Quadrupedal Locomotion: Coupled Planning, Terrain Mapping and Whole-Body Control
• RLOC: Terrain-Aware Legged Locomotion using Reinforcement Learning and Optimal Control
• TAMOLS: Terrain-Aware Motion Optimization for Legged Systems

Cited by in corpus (8)

• Grasp Multiple Objects with One Hand
• Learning Whole-Body Loco-Manipulation for Omni-Directional Task Space Pose Tracking with a Wheeled-Quadrupedal-Manipulator
• Deep Reinforcement Learning for Bipedal Locomotion: A Brief Survey
• FR-Net: Learning Robust Quadrupedal Fall Recovery on Challenging Terrains through Mass-Contact Prediction
• Hybrid Dynamics Modeling and Trajectory Planning for a Cable-Trailer System with a Quadruped Robot
• Exploiting Hybrid Policy in Reinforcement Learning for Interpretable Temporal Logic Manipulation
• Advances, challenges, and opportunities for legged robots
• Agile perceptive multi-skill locomotion for quadrupedal robots in the wild