Learning Fast Adaptation with Meta Strategy Optimization
arXiv:1909.12995
Abstract
The ability to walk in new scenarios is a key milestone on the path toward real-world applications of legged robots. In this work, we introduce Meta Strategy Optimization, a meta-learning algorithm for training policies with latent variable inputs that can quickly adapt to new scenarios with a handful of trials in the target environment. The key idea behind MSO is to expose the same adaptation process, Strategy Optimization (SO), to both the training and testing phases. This allows MSO to effectively learn locomotion skills as well as a latent space that is suitable for fast adaptation. We evaluate our method on a real quadruped robot and demonstrate successful adaptation in various scenarios, including sim-to-real transfer, walking with a weakened motor, or climbing up a slope. Furthermore, we quantitatively analyze the generalization capability of the trained policy in simulated environments. Both real and simulated experiments show that our method outperforms previous methods in adaptation to novel tasks.
References in corpus (10)
- Continuous control with deep reinforcement learning
- Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
- Learning agile and dynamic motor skills for legged robots
- RL: Fast Reinforcement Learning via Slow Reinforcement Learning
- Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World
- Simple random search provides a competitive approach to reinforcement learning
- Evolved Policy Gradients
- Meta-Learning and Universality: Deep Representations and Gradient Descent can Approximate any Learning Algorithm
- Policy Transfer with Strategy Optimization
- Data-Efficient Learning for Sim-to-Real Robotic Grasping using Deep Point Cloud Prediction Networks