Importance Weighted Evolution Strategies
arXiv:1811.04624
Abstract
Evolution Strategies (ES) emerged as a scalable alternative to popular Reinforcement Learning (RL) techniques, providing an almost perfect speedup when distributed across hundreds of CPU cores thanks to a reduced communication overhead. Despite providing large improvements in wall-clock time, ES is data inefficient when compared to competing RL methods. One of the main causes of such inefficiency is the collection of large batches of experience, which are discarded after each policy update. In this work, we study how to perform more than one update per batch of experience by means of Importance Sampling while preserving the scalability of the original method. The proposed method, Importance Weighted Evolution Strategies (IW-ES), shows promising results and is a first step towards designing efficient ES algorithms.
arXiv commentNIPS Deep Reinforcement Learning Workshop 2018
References in corpus (8)
- Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
- Evolution Strategies as a Scalable Alternative to Reinforcement Learning
- Emergence of Locomotion Behaviours in Rich Environments
- Revisiting Distributed Synchronous SGD
- IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures
- Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning
- Evolved Policy Gradients
- Multi-task Deep Reinforcement Learning with PopArt