A new Taxonomy of Continuous Global Optimization Algorithms

Abstract

Surrogate-based optimization, nature-inspired metaheuristics, and hybrid combinations have become state of the art in algorithm design for solving real-world optimization problems. Still, it is difficult for practitioners to get an overview that explains their advantages in comparison to a large number of available methods in the scope of optimization. Available taxonomies lack the embedding of current approaches in the larger context of this broad field. This article presents a taxonomy of the field, which explores and matches algorithm strategies by extracting similarities and differences in their search strategies. A particular focus lies on algorithms using surrogates, nature-inspired designs, and those created by design optimization. The extracted features of components or operators allow us to create a set of classification indicators to distinguish between a small number of classes. The features allow a deeper understanding of components of the search strategies and further indicate the close connections between the different algorithm designs. We present intuitive analogies to explain the basic principles of the search algorithms, particularly useful for novices in this research field. Furthermore, this taxonomy allows recommendations for the applicability of the corresponding algorithms.

35 pages total, 28 written pages, 4 figures, 2019 Reworked Version

References in corpus (11)

• Deep Learning in Neural Networks: An Overview
• Practical Bayesian Optimization of Machine Learning Algorithms
• A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning
• A Survey of Neuromorphic Computing and Neural Networks in Hardware
• Automated Algorithm Selection on Continuous Black-Box Problems By Combining Exploratory Landscape Analysis and Machine Learning
• SPOT: An R Package For Automatic and Interactive Tuning of Optimization Algorithms by Sequential Parameter Optimization
• Initial Design Strategies and their Effects on Sequential Model-Based Optimization
• Online Selection of CMA-ES Variants
• Data-efficient Neuroevolution with Kernel-Based Surrogate Models
• Surrogate Models for Enhancing the Efficiency of Neuroevolution in Reinforcement Learning
• Gray-box optimization and factorized distribution algorithms: where two worlds collide

Cited by in corpus (5)

• Benchmarking in Optimization: Best Practice and Open Issues
• Parallel Hyperparameter Optimization Of Spiking Neural Network
• Evolving Continuous Optimisers from Scratch
• Metaheuristics for (Variable-Size) Mixed Optimization Problems: A Unified Taxonomy and Survey
• Cognitive Capabilities for the CAAI in Cyber-Physical Production Systems