Bayesian Optimization with Adaptive Kernels for Robot Control

Abstract

Active policy search combines the trial-and-error methodology from policy search with Bayesian optimization to actively find the optimal policy. First, policy search is a type of reinforcement learning which has become very popular for robot control, for its ability to deal with complex continuous state and action spaces. Second, Bayesian optimization is a sample efficient global optimization method that uses a surrogate model, like a Gaussian process, and optimal decision making to carefully select each sample during the optimization process. Sample efficiency is of paramount importance when each trial involves the real robot, expensive Monte Carlo runs, or a complex simulator. Black-box Bayesian optimization generally assumes a cost function from a stationary process, because nonstationary modeling is usually based on prior knowledge. However, many control problems are inherently nonstationary due to their failure conditions, terminal states and other abrupt effects. In this paper, we present a kernel function specially designed for Bayesian optimization, that allows nonstationary modeling without prior knowledge, using an adaptive local region. The new kernel results in an improved local search (exploitation), without penalizing the global search (exploration), as shown experimentally in well-known optimization benchmarks and robot control scenarios. We finally show its potential for the design of the wing shape of a UAV.

2017 IEEE International Conference on Robotics and Automation (ICRA). arXiv admin note: substantial text overlap with arXiv:1610.00366

References in corpus (6)

• Practical Bayesian Optimization of Machine Learning Algorithms
• A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning
• Robots that can adapt like animals
• Predictive Entropy Search for Efficient Global Optimization of Black-box Functions
• BayesOpt: A Bayesian Optimization Library for Nonlinear Optimization, Experimental Design and Bandits
• Heteroscedastic Treed Bayesian Optimisation

Cited by in corpus (6)

• Practical Bayesian optimization in the presence of outliers
• Bayesian Optimization Meets Riemannian Manifolds in Robot Learning
• PHOENICS: A universal deep Bayesian optimizer
• Bayesian Optimization that Limits Search Region to Lower Dimensions Utilizing Local GPR
• Robust Bayesian Optimization with Student-t Likelihood
• Neural fidelity warping for efficient robot morphology design