Sample Efficient Optimization for Learning Controllers for Bipedal Locomotion
arXiv:1610.04795 · doi:10.1109/HUMANOIDS.2016.7803249
Abstract
Learning policies for bipedal locomotion can be difficult, as experiments are expensive and simulation does not usually transfer well to hardware. To counter this, we need al- gorithms that are sample efficient and inherently safe. Bayesian Optimization is a powerful sample-efficient tool for optimizing non-convex black-box functions. However, its performance can degrade in higher dimensions. We develop a distance metric for bipedal locomotion that enhances the sample-efficiency of Bayesian Optimization and use it to train a 16 dimensional neuromuscular model for planar walking. This distance metric reflects some basic gait features of healthy walking and helps us quickly eliminate a majority of unstable controllers. With our approach we can learn policies for walking in less than 100 trials for a range of challenging settings. In simulation, we show results on two different costs and on various terrains including rough ground and ramps, sloping upwards and downwards. We also perturb our models with unknown inertial disturbances analogous with differences between simulation and hardware. These results are promising, as they indicate that this method can potentially be used to learn control policies on hardware.
To appear in International Conference on Humanoid Robots (Humanoids '2016), IEEE-RAS. (Rika Antonova and Akshara Rai contributed equally)
References in corpus (1)
Cited by in corpus (7)
- On the Design of LQR Kernels for Efficient Controller Learning
- Robot Learning with Crash Constraints
- Beyond Basins of Attraction: Quantifying Robustness of Natural Dynamics
- Deep Kernels for Optimizing Locomotion Controllers
- Learning Predictive Models for Ergonomic Control of Prosthetic Devices
- Efficient Model Identification for Tensegrity Locomotion
- Enhancement of Energy-Based Swing-Up Controller via Entropy Search