Learning Convex Optimization Control Policies
arXiv:1912.09529
Abstract
Many control policies used in various applications determine the input or action by solving a convex optimization problem that depends on the current state and some parameters. Common examples of such convex optimization control policies (COCPs) include the linear quadratic regulator (LQR), convex model predictive control (MPC), and convex control-Lyapunov or approximate dynamic programming (ADP) policies. These types of control policies are tuned by varying the parameters in the optimization problem, such as the LQR weights, to obtain good performance, judged by application-specific metrics. Tuning is often done by hand, or by simple methods such as a crude grid search. In this paper we propose a method to automate this process, by adjusting the parameters using an approximate gradient of the performance metric with respect to the parameters. Our method relies on recently developed methods that can efficiently evaluate the derivative of the solution of a convex optimization problem with respect to its parameters. We illustrate our method on several examples.
Authors listed in alphabetical order
References in corpus (12)
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- Continuous control with deep reinforcement learning
- TensorFlow: A system for large-scale machine learning
- On the difficulty of training Recurrent Neural Networks
- OSQP: An Operator Splitting Solver for Quadratic Programs
- Evolution Strategies as a Scalable Alternative to Reinforcement Learning
- Embedded Online Optimization for Model Predictive Control at Megahertz Rates
- Input Convex Neural Networks
- High-Speed Finite Control Set Model Predictive Control for Power Electronics
- Differentiable Convex Optimization Layers
- On the Differentiability of the Solution to Convex Optimization Problems
- Path Integral Networks: End-to-End Differentiable Optimal Control