Towards Large Scale Automated Algorithm Design by Integrating Modular Benchmarking Frameworks

Abstract

We present a first proof-of-concept use-case that demonstrates the efficiency of interfacing the algorithm framework ParadisEO with the automated algorithm configuration tool irace and the experimental platform IOHprofiler. By combing these three tools, we obtain a powerful benchmarking environment that allows us to systematically analyze large classes of algorithms on complex benchmark problems. Key advantages of our pipeline are fast evaluation times, the possibility to generate rich data sets to support the analysis of the algorithms, and a standardized interface that can be used to benchmark very broad classes of sampling-based optimization heuristics. In addition to enabling systematic algorithm configuration studies, our approach paves a way for assessing the contribution of new ideas in interplay with already existing operators -- a promising avenue for our research domain, which at present may have a too strong focus on comparing entire algorithm instances.

arXiv commentTo appear in the Companion Material of ACM Genetic and Evolutionary Computation Conference (GECCO'21) as workshop paper

References in corpus (4)

• COCO: A Platform for Comparing Continuous Optimizers in a Black-Box Setting
• Benchmarking Discrete Optimization Heuristics with IOHprofiler
• Benchmarking a $(\mu +\lambda )$ Genetic Algorithm with Configurable Crossover Probability
• Continuous Optimization Benchmarks by Simulation

Cited by in corpus (3)

• Paradiseo: From a Modular Framework for Evolutionary Computation to the Automated Design of Metaheuristics ---22 Years of Paradiseo---
• Automated Configuration of Genetic Algorithms by Tuning for Anytime Performance
• Improving Nevergrad's Algorithm Selection Wizard NGOpt through Automated Algorithm Configuration