On the Design of LQR Kernels for Efficient Controller Learning

Abstract

Finding optimal feedback controllers for nonlinear dynamic systems from data is hard. Recently, Bayesian optimization (BO) has been proposed as a powerful framework for direct controller tuning from experimental trials. For selecting the next query point and finding the global optimum, BO relies on a probabilistic description of the latent objective function, typically a Gaussian process (GP). As is shown herein, GPs with a common kernel choice can, however, lead to poor learning outcomes on standard quadratic control problems. For a first-order system, we construct two kernels that specifically leverage the structure of the well-known Linear Quadratic Regulator (LQR), yet retain the flexibility of Bayesian nonparametric learning. Simulations of uncertain linear and nonlinear systems demonstrate that the LQR kernels yield superior learning performance.

8 pages, 5 figures, to appear in 56th IEEE Conference on Decision and Control (CDC 2017)

References in corpus (1)

• Virtual vs. Real: Trading Off Simulations and Physical Experiments in Reinforcement Learning with Bayesian Optimization

Cited by in corpus (10)

• Data-efficient Auto-tuning with Bayesian Optimization: An Industrial Control Study
• On Controller Tuning with Time-Varying Bayesian Optimization
• Deep Reinforcement Learning for Event-Triggered Control
• Global Search with Bernoulli Alternation Kernel for Task-oriented Grasping Informed by Simulation
• Controller Design via Experimental Exploration with Robustness Guarantees
• Bayesian Optimization Meets Riemannian Manifolds in Robot Learning
• A Multi-Fidelity Bayesian Approach to Safe Controller Design
• Linearly-constrained Linear Quadratic Regulator from the viewpoint of kernel methods
• High-Dimensional Bayesian Optimization via Nested Riemannian Manifolds
• Deep Reinforcement Learning with Linear Quadratic Regulator Regions