Neural Generative Models for Global Optimization with Gradients

Abstract

The aim of global optimization is to find the global optimum of arbitrary classes of functions, possibly highly multimodal ones. In this paper we focus on the subproblem of global optimization for differentiable functions and we propose an Evolutionary Search-inspired solution where we model point search distributions via Generative Neural Networks. This approach enables us to model diverse and complex search distributions based on which we can efficiently explore complicated objective landscapes. In our experiments we show the practical superiority of our algorithm versus classical Evolutionary Search and gradient-based solutions on a benchmark set of multimodal functions, and demonstrate how it can be used to accelerate Bayesian Optimization with Gaussian Processes.

References in corpus (5)

• Practical Bayesian Optimization of Machine Learning Algorithms
• Evolution Strategies as a Scalable Alternative to Reinforcement Learning
• Gradient-based Hyperparameter Optimization through Reversible Learning
• Practical Bayesian Optimization for Model Fitting with Bayesian Adaptive Direct Search
• Bayesian Optimization with Gradients

Cited by in corpus (2)

• Inverse Design of Grating Couplers Using the Policy Gradient Method from Reinforcement Learning
• Improving Evolutionary Strategies with Generative Neural Networks