Robust Policy Search for Robot Navigation

Abstract

Complex robot navigation and control problems can be framed as policy search problems. However, interactive learning in uncertain environments can be expensive, requiring the use of data-efficient methods. Bayesian optimization is an efficient nonlinear optimization method where queries are carefully selected to gather information about the optimum location. This is achieved by a surrogate model, which encodes past information, and the acquisition function for query selection. Bayesian optimization can be very sensitive to uncertainty in the input data or prior assumptions. In this work, we incorporate both robust optimization and statistical robustness, showing that both types of robustness are synergistic. For robust optimization we use an improved version of unscented Bayesian optimization which provides safe and repeatable policies in the presence of policy uncertainty. We also provide new theoretical insights. For statistical robustness, we use an adaptive surrogate model and we introduce the Boltzmann selection as a stochastic acquisition method to have convergence guarantees and improved performance even with surrogate modeling errors. We present results in several optimization benchmarks and robot tasks.

References in corpus (11)

• Practical Bayesian Optimization of Machine Learning Algorithms
• Robots that can adapt like animals
• Batch Bayesian Optimization via Local Penalization
• Max-value Entropy Search for Efficient Bayesian Optimization
• Batched Large-scale Bayesian Optimization in High-dimensional Spaces
• Parallel Predictive Entropy Search for Batch Global Optimization of Expensive Objective Functions
• Adversarially Robust Optimization with Gaussian Processes
• Bayesian Optimization with Adaptive Kernels for Robot Control
• Heteroscedastic Treed Bayesian Optimisation
• Bayesian optimisation under uncertain inputs
• Finding safe 3D robot grasps through efficient haptic exploration with unscented Bayesian optimization and collision penalty