MPC-Net: A First Principles Guided Policy Search

Abstract

We present an Imitation Learning approach for the control of dynamical systems with a known model. Our policy search method is guided by solutions from MPC. Typical policy search methods of this kind minimize a distance metric between the guiding demonstrations and the learned policy. Our loss function, however, corresponds to the minimization of the control Hamiltonian, which derives from the principle of optimality. Therefore, our algorithm directly attempts to solve the optimality conditions with a parameterized class of control laws. Additionally, the proposed loss function explicitly encodes the constraints of the optimal control problem and we provide numerical evidence that its minimization achieves improved constraint satisfaction. We train a mixture-of-expert neural network architecture for controlling a quadrupedal robot and show that this policy structure is well suited for such multimodal systems. The learned policy can successfully stabilize different gaits on the real walking robot from less than 10 min of demonstration data.

References in corpus (5)

• Learning agile and dynamic motor skills for legged robots
• An Algorithmic Perspective on Imitation Learning
• Whole-Body Nonlinear Model Predictive Control Through Contacts for Quadrupeds
• An Efficient Optimal Planning and Control Framework For Quadrupedal Locomotion
• Real-Time Motion Planning of Legged Robots: A Model Predictive Control Approach

Cited by in corpus (9)

• Contraction Theory for Nonlinear Stability Analysis and Learning-based Control: A Tutorial Overview
• Learning Free Gait Transition for Quadruped Robots via Phase-Guided Controller
• Imitation Learning from MPC for Quadrupedal Multi-Gait Control
• Learning-based Robust Motion Planning with Guaranteed Stability: A Contraction Theory Approach
• Imitating and Finetuning Model Predictive Control for Robust and Symmetric Quadrupedal Locomotion
• GRiD: GPU-Accelerated Rigid Body Dynamics with Analytical Gradients
• Approximate non-linear model predictive control with safety-augmented neural networks
• Guided Policy Search using Sequential Convex Programming for Initialization of Trajectory Optimization Algorithms
• The Mini Wheelbot: A Testbed for Learning-based Balancing, Flips, and Articulated Driving