Neural-iLQR: A Learning-Aided Shooting Method for Trajectory Optimization
arXiv:2011.10737
Abstract
Iterative linear quadratic regulator (iLQR) has gained wide popularity in addressing trajectory optimization problems with nonlinear system models. However, as a model-based shooting method, it relies heavily on an accurate system model to update the optimal control actions and the trajectory determined with forward integration, thus becoming vulnerable to inevitable model inaccuracies. Recently, substantial research efforts in learning-based methods for optimal control problems have been progressing significantly in addressing unknown system models, particularly when the system has complex interactions with the environment. Yet a deep neural network is normally required to fit substantial scale of sampling data. In this work, we present Neural-iLQR, a learning-aided shooting method over the unconstrained control space, in which a neural network with a simple structure is used to represent the local system model. In this framework, the trajectory optimization task is achieved with simultaneous refinement of the optimal policy and the neural network iteratively, without relying on the prior knowledge of the system model. Through comprehensive evaluations on two illustrative control tasks, the proposed method is shown to outperform the conventional iLQR significantly in the presence of inaccuracies in system models.
arXiv comment7 pages, 7 figures
References in corpus (8)
- Deep Reinforcement Learning for Multi-Agent Systems: A Review of Challenges, Solutions and Applications
- Collective Robot Reinforcement Learning with Distributed Asynchronous Guided Policy Search
- Model-Based Reinforcement Learning via Meta-Policy Optimization
- Evolution-Guided Policy Gradient in Reinforcement Learning
- Walking Posture Adaptation for Legged Robot Navigation in Confined Spaces
- Curious iLQR: Resolving Uncertainty in Model-based RL
- Alternating Direction Method of Multipliers for Constrained Iterative LQR in Autonomous Driving
- D2C 2.0: Decoupled Data-Based Approach for Learning to Control Stochastic Nonlinear Systems via Model-Free ILQR