Safe Controller Optimization for Quadrotors with Gaussian Processes

Abstract

One of the most fundamental problems when designing controllers for dynamic systems is the tuning of the controller parameters. Typically, a model of the system is used to obtain an initial controller, but ultimately the controller parameters must be tuned manually on the real system to achieve the best performance. To avoid this manual tuning step, methods from machine learning, such as Bayesian optimization, have been used. However, as these methods evaluate different controller parameters on the real system, safety-critical system failures may happen. In this paper, we overcome this problem by applying, for the first time, a recently developed safe optimization algorithm, SafeOpt, to the problem of automatic controller parameter tuning. Given an initial, low-performance controller, SafeOpt automatically optimizes the parameters of a control law while guaranteeing safety. It models the underlying performance measure as a Gaussian process and only explores new controller parameters whose performance lies above a safe performance threshold with high probability. Experimental results on a quadrotor vehicle indicate that the proposed method enables fast, automatic, and safe optimization of controller parameters without human intervention.

IEEE International Conference on Robotics and Automation, 2016. 6 pages, 4 figures. A video of the experiments can be found at http://tiny.cc/icra16_video . A Python implementation of the algorithm is available at https://github.com/befelix/SafeOpt

References in corpus (3)

• Bayesian Optimization with Unknown Constraints
• Automatic LQR Tuning Based on Gaussian Process Global Optimization
• Safe Exploration in Markov Decision Processes

Cited by in corpus (67)

• Neural Lander: Stable Drone Landing Control using Learned Dynamics
• Safe Learning of Regions of Attraction for Uncertain, Nonlinear Systems with Gaussian Processes
• Automatic LQR Tuning Based on Gaussian Process Global Optimization
• Data-efficient Auto-tuning with Bayesian Optimization: An Industrial Control Study
• Safe Exploration in Finite Markov Decision Processes with Gaussian Processes
• Learning for Safety-Critical Control with Control Barrier Functions
• Performance-Driven Cascade Controller Tuning with Bayesian Optimization
• Stagewise Safe Bayesian Optimization with Gaussian Processes
• Control Barriers in Bayesian Learning of System Dynamics
• Real-time System Identification Using Deep Learning for Linear Processes with Application to Unmanned Aerial Vehicles
• Data-Driven Multi-Objective Controller Optimization for a Magnetically-Levitated Nanopositioning System
• Episodic Learning with Control Lyapunov Functions for Uncertain Robotic Systems
• A Control Barrier Perspective on Episodic Learning via Projection-to-State Safety
• Safe Reinforcement Learning via Curriculum Induction
• Safe global optimization of expensive noisy black-box functions in the $\delta$-Lipschitz framework
• Goal-Driven Dynamics Learning via Bayesian Optimization
• ${\mathcal{L}}_1$ Adaptive Augmentation for Geometric Tracking Control of Quadrotors
• Probabilistic Safety Constraints for Learned High Relative Degree System Dynamics
• Robot Learning with Crash Constraints
• Contextual Tuning of Model Predictive Control for Autonomous Racing
• On the Design of LQR Kernels for Efficient Controller Learning
• Gait learning for soft microrobots controlled by light fields
• VisuoSpatial Foresight for Multi-Step, Multi-Task Fabric Manipulation
• DiffTune-MPC: Closed-Loop Learning for Model Predictive Control
• Safe Learning and Optimization Techniques: Towards a Survey of the State of the Art
• Advanced Manufacturing Configuration by Sample-efficient Batch Bayesian Optimization
• Uniform Error and Posterior Variance Bounds for Gaussian Process Regression with Application to Safe Control
• Controller Design via Experimental Exploration with Robustness Guarantees
• Multi-Agent Safe Planning with Gaussian Processes
• Posterior Variance Analysis of Gaussian Processes with Application to Average Learning Curves
• A Control Lyapunov Perspective on Episodic Learning via Projection to State Stability
• Plasma Spray Process Parameters Configuration using Sample-efficient Batch Bayesian Optimization
• BOATS: Bayesian Optimization for Active Control of ThermoacousticS
• VisuoSpatial Foresight for Physical Sequential Fabric Manipulation
• Better safe than sorry: Risky function exploitation through safe optimization
• Robust Regression for Safe Exploration in Control
• Preference-based MPC calibration
• Design of Deep Neural Networks as Add-on Blocks for Improving Impromptu Trajectory Tracking
• Excursion Search for Constrained Bayesian Optimization under a Limited Budget of Failures
• Cautious Bayesian Optimization for Efficient and Scalable Policy Search
• Second-Order Sampling-Based Stability Guarantee for Data-Driven Control Systems
• A Learnable Safety Measure
• Stable Reinforcement Learning with Unbounded State Space
• ROMO: Retrieval-enhanced Offline Model-based Optimization
• The Mini Wheelbot: A Testbed for Learning-based Balancing, Flips, and Articulated Driving
• Safe Policy Search with Gaussian Process Models
• Performance-driven Constrained Optimal Auto-Tuner for MPC
• Constrained Discrete Black-Box Optimization using Mixed-Integer Programming
• Uncertainty-aware Safe Exploratory Planning using Gaussian Process and Neural Control Contraction Metric
• AutoTune: Controller Tuning for High-Speed Flight
• Neural Lyapunov Redesign
• Reference design for closed loop system optimization
• Simulation-Aided Policy Tuning for Black-Box Robot Learning
• Are Evolutionary Algorithms Safe Optimizers?
• Protective Policy Transfer
• Classified Regression for Bayesian Optimization: Robot Learning with Unknown Penalties
• Experience Recommendation for Long Term Safe Learning-based Model Predictive Control in Changing Operating Conditions
• A Probabilistic Interpretation of Self-Paced Learning with Applications to Reinforcement Learning
• C-GLISp: Preference-Based Global Optimization under Unknown Constraints with Applications to Controller Calibration
• Learning To Estimate Regions Of Attraction Of Autonomous Dynamical Systems Using Physics-Informed Neural Networks
• Cross-Platform Learnable Fuzzy Gain-Scheduled Proportional-Integral-Derivative Controller Tuning via Physics-Constrained Meta-Learning and Reinforcement Learning Adaptation
• Heteroscedastic Bayesian Optimization-Based Dynamic PID Tuning for Accurate and Robust UAV Trajectory Tracking
• Enhancement of Energy-Based Swing-Up Controller via Entropy Search
• Online Parameter Estimation for Safety-Critical Systems with Gaussian Processes
• Bayesian optimization for modular black-box systems with switching costs
• The Impact of Data on the Stability of Learning-Based Control- Extended Version
• ROIAL: Region of Interest Active Learning for Characterizing Exoskeleton Gait Preference Landscapes