On the Design of LQR Kernels for Efficient Controller Learning
arXiv:1709.07089 · doi:10.1109/CDC.2017.8264429
Abstract
Finding optimal feedback controllers for nonlinear dynamic systems from data is hard. Recently, Bayesian optimization (BO) has been proposed as a powerful framework for direct controller tuning from experimental trials. For selecting the next query point and finding the global optimum, BO relies on a probabilistic description of the latent objective function, typically a Gaussian process (GP). As is shown herein, GPs with a common kernel choice can, however, lead to poor learning outcomes on standard quadratic control problems. For a first-order system, we construct two kernels that specifically leverage the structure of the well-known Linear Quadratic Regulator (LQR), yet retain the flexibility of Bayesian nonparametric learning. Simulations of uncertain linear and nonlinear systems demonstrate that the LQR kernels yield superior learning performance.
arXiv comment8 pages, 5 figures, to appear in 56th IEEE Conference on Decision and Control (CDC 2017)
References in corpus (4)
- Safe Controller Optimization for Quadrotors with Gaussian Processes
- Virtual vs. Real: Trading Off Simulations and Physical Experiments in Reinforcement Learning with Bayesian Optimization
- Automatic LQR Tuning Based on Gaussian Process Global Optimization
- Sample Efficient Optimization for Learning Controllers for Bipedal Locomotion
Cited by in corpus (10)
- Data-efficient Auto-tuning with Bayesian Optimization: An Industrial Control Study
- On Controller Tuning with Time-Varying Bayesian Optimization
- Deep Reinforcement Learning for Event-Triggered Control
- Global Search with Bernoulli Alternation Kernel for Task-oriented Grasping Informed by Simulation
- Controller Design via Experimental Exploration with Robustness Guarantees
- Bayesian Optimization Meets Riemannian Manifolds in Robot Learning
- A Multi-Fidelity Bayesian Approach to Safe Controller Design
- Linearly-constrained Linear Quadratic Regulator from the viewpoint of kernel methods
- High-Dimensional Bayesian Optimization via Nested Riemannian Manifolds
- Deep Reinforcement Learning with Linear Quadratic Regulator Regions