Automatic LQR Tuning Based on Gaussian Process Global Optimization

Abstract

This paper proposes an automatic controller tuning framework based on linear optimal control combined with Bayesian optimization. With this framework, an initial set of controller gains is automatically improved according to a pre-defined performance objective evaluated from experimental data. The underlying Bayesian optimization algorithm is Entropy Search, which represents the latent objective as a Gaussian process and constructs an explicit belief over the location of the objective minimum. This is used to maximize the information gain from each experimental evaluation. Thus, this framework shall yield improved controllers with fewer evaluations compared to alternative approaches. A seven-degree-of-freedom robot arm balancing an inverted pole is used as the experimental demonstrator. Results of a two- and four-dimensional tuning problems highlight the method's potential for automatic controller tuning on robotic platforms.

8 pages, 5 figures, to appear in IEEE 2016 International Conference on Robotics and Automation. Video demonstration of the experiments available at https://am.is.tuebingen.mpg.de/publications/marco_icra_2016

References in corpus (1)

• Safe Controller Optimization for Quadrotors with Gaussian Processes

Cited by in corpus (19)

• Safe Controller Optimization for Quadrotors with Gaussian Processes
• Virtual vs. Real: Trading Off Simulations and Physical Experiments in Reinforcement Learning with Bayesian Optimization
• Data-efficient Auto-tuning with Bayesian Optimization: An Industrial Control Study
• Automated Controller Calibration by Kalman Filtering
• On the Design of LQR Kernels for Efficient Controller Learning
• Robot Learning with Crash Constraints
• Benchmarking Potential Based Rewards for Learning Humanoid Locomotion
• Gait learning for soft microrobots controlled by light fields
• On Controller Tuning with Time-Varying Bayesian Optimization
• Controller Design via Experimental Exploration with Robustness Guarantees
• Automatic Gain Tuning of a Momentum Based Balancing Controller for Humanoid Robots
• Local Bayesian Optimization for Controller Tuning with Crash Constraints
• Benchmarking Structured Policies and Policy Optimization for Real-World Dexterous Object Manipulation
• Event-triggered Learning for Linear Quadratic Control
• The Mini Wheelbot: A Testbed for Learning-based Balancing, Flips, and Articulated Driving
• AutoTune: Controller Tuning for High-Speed Flight
• PACSBO: Probably approximately correct safe Bayesian optimization
• Simulation-Aided Policy Tuning for Black-Box Robot Learning
• Minimisation of Polyak-Łojasewicz Functions Using Random Zeroth-Order Oracles