ES-ENAS: Efficient Evolutionary Optimization for Large Hybrid Search Spaces

Abstract

In this paper, we approach the problem of optimizing blackbox functions over large hybrid search spaces consisting of both combinatorial and continuous parameters. We demonstrate that previous evolutionary algorithms which rely on mutation-based approaches, while flexible over combinatorial spaces, suffer from a curse of dimensionality in high dimensional continuous spaces both theoretically and empirically, which thus limits their scope over hybrid search spaces as well. In order to combat this curse, we propose ES-ENAS, a simple and modular joint optimization procedure combining the class of sample-efficient smoothed gradient techniques, commonly known as Evolutionary Strategies (ES), with combinatorial optimizers in a highly scalable and intuitive way, inspired by the one-shot or supernet paradigm introduced in Efficient Neural Architecture Search (ENAS). By doing so, we achieve significantly more sample efficiency, which we empirically demonstrate over synthetic benchmarks, and are further able to apply ES-ENAS for architecture search over popular RL benchmarks.

arXiv commentPreviously published at ICLR 2020 NAS Workshop. See https://github.com/google-research/google-research/tree/master/es_enas for associated code

References in corpus (13)

• Neural Architecture Search with Reinforcement Learning
• Trust Region Policy Optimization
• DARTS: Differentiable Architecture Search
• Compressing Neural Networks with the Hashing Trick
• Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning
• The Evolved Transformer
• Simple random search provides a competitive approach to reinforcement learning
• AutoML-Zero: Evolving Machine Learning Algorithms From Scratch
• Orthogonal Random Features
• Gradientless Descent: High-Dimensional Zeroth-Order Optimization
• PyGlove: Symbolic Programming for Automated Machine Learning
• Non-Differentiable Supervised Learning with Evolution Strategies and Hybrid Methods
• Robotic Table Tennis with Model-Free Reinforcement Learning