Bayesian Optimization in Variational Latent Spaces with Dynamic Compression
arXiv:1907.04796
Abstract
Data-efficiency is crucial for autonomous robots to adapt to new tasks and environments. In this work we focus on robotics problems with a budget of only 10-20 trials. This is a very challenging setting even for data-efficient approaches like Bayesian optimization (BO), especially when optimizing higher-dimensional controllers. Simulated trajectories can be used to construct informed kernels for BO. However, previous work employed supervised ways of extracting low-dimensional features for these. We propose a model and architecture for a sequential variational autoencoder that embeds the space of simulated trajectories into a lower-dimensional space of latent paths in an unsupervised way. We further compress the search space for BO by reducing exploration in parts of the state space that are undesirable, without requiring explicit constraints on controller parameters. We validate our approach with hardware experiments on a Daisy hexapod robot and an ABB Yumi manipulator. We also present simulation experiments with further comparisons to several baselines on Daisy and two manipulators. Our experiments indicate the proposed trajectory-based kernel with dynamic compression can offer ultra data-efficient optimization.
(Rika Antonova and Akshara Rai contributed equally)
References in corpus (10)
- An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling
- Semi-Supervised Learning with Deep Generative Models
- Robots that can adapt like animals
- State Representation Learning for Control: An Overview
- Learning Dexterous In-Hand Manipulation
- Disentangled Sequential Autoencoder
- Learning to Walk via Deep Reinforcement Learning
- Feedback Control of a Cassie Bipedal Robot: Walking, Standing, and Riding a Segway
- Variational Inference for Data-Efficient Model Learning in POMDPs
- Using Deep Reinforcement Learning to Learn High-Level Policies on the ATRIAS Biped