Landscape-Aware Fixed-Budget Performance Regression and Algorithm Selection for Modular CMA-ES Variants

Abstract

Automated algorithm selection promises to support the user in the decisive task of selecting a most suitable algorithm for a given problem. A common component of these machine-trained techniques are regression models which predict the performance of a given algorithm on a previously unseen problem instance. In the context of numerical black-box optimization, such regression models typically build on exploratory landscape analysis (ELA), which quantifies several characteristics of the problem. These measures can be used to train a supervised performance regression model. First steps towards ELA-based performance regression have been made in the context of a fixed-target setting. In many applications, however, the user needs to select an algorithm that performs best within a given budget of function evaluations. Adopting this fixed-budget setting, we demonstrate that it is possible to achieve high-quality performance predictions with off-the-shelf supervised learning approaches, by suitably combining two differently trained regression models. We test this approach on a very challenging problem: algorithm selection on a portfolio of very similar algorithms, which we choose from the family of modular CMA-ES algorithms.

To appear in Proc. of Genetic and Evolutionary Computation Conference (GECCO'20)

References in corpus (5)

• Scikit-learn: Machine Learning in Python
• SATzilla: Portfolio-based Algorithm Selection for SAT
• Automated Algorithm Selection on Continuous Black-Box Problems By Combining Exploratory Landscape Analysis and Machine Learning
• Evolving the Structure of Evolution Strategies
• Towards White-box Benchmarks for Algorithm Control

Cited by in corpus (9)

• Towards Explainable Exploratory Landscape Analysis: Extreme Feature Selection for Classifying BBOB Functions
• The Impact of Hyper-Parameter Tuning for Landscape-Aware Performance Regression and Algorithm Selection
• Benchmarking Feature-based Algorithm Selection Systems for Black-box Numerical Optimization
• Personalizing Performance Regression Models to Black-Box Optimization Problems
• Towards Exploratory Landscape Analysis for Large-scale Optimization: A Dimensionality Reduction Framework
• Automated Algorithm Selection for Radar Network Configuration
• A Survey of Features Used for Representing Black-box Single-objective Continuous Optimization
• Does Dimensionality Reduction via Random Projections Preserve Landscape Features?
• On Constructing Algorithm Portfolios in Algorithm Selection for Computationally Expensive Black-box Optimization in the Fixed-budget Setting