COCO: A Platform for Comparing Continuous Optimizers in a Black-Box Setting

Abstract

We introduce COCO, an open source platform for Comparing Continuous Optimizers in a black-box setting. COCO aims at automatizing the tedious and repetitive task of benchmarking numerical optimization algorithms to the greatest possible extent. The platform and the underlying methodology allow to benchmark in the same framework deterministic and stochastic solvers for both single and multiobjective optimization. We present the rationales behind the (decade-long) development of the platform as a general proposition for guidelines towards better benchmarking. We detail underlying fundamental concepts of COCO such as the definition of a problem as a function instance, the underlying idea of instances, the use of target values, and runtime defined by the number of function calls as the central performance measure. Finally, we give a quick overview of the basic code structure and the currently available test suites.

Optimization Methods and Software, Taylor & Francis, In press, pp.1-31

References in corpus (1)

• Best practices for comparing optimization algorithms

Cited by in corpus (37)

• Towards Explainable Exploratory Landscape Analysis: Extreme Feature Selection for Classifying BBOB Functions
• Paradiseo: From a Modular Framework for Evolutionary Computation to the Automated Design of Metaheuristics ---22 Years of Paradiseo---
• The importance of being constrained: dealing with infeasible solutions in Differential Evolution and beyond
• The Impact of Hyper-Parameter Tuning for Landscape-Aware Performance Regression and Algorithm Selection
• CMA-ES with Learning Rate Adaptation
• Benchmarking Feature-based Algorithm Selection Systems for Black-box Numerical Optimization
• Neural-BO: A Black-box Optimization Algorithm using Deep Neural Networks
• CMA-ES with Learning Rate Adaptation: Can CMA-ES with Default Population Size Solve Multimodal and Noisy Problems?
• Comparing Algorithm Selection Approaches on Black-Box Optimization Problems
• Fixed-Target Runtime Analysis
• Fast Mutation in Crossover-based Algorithms
• ITSO: A novel Inverse Transform Sampling-based Optimization algorithm for stochastic search
• A Local Optima Network View of Real Function Fitness Landscapes
• Optimizing with Low Budgets: a Comparison on the Black-box Optimization Benchmarking Suite and OpenAI Gym
• Towards Large Scale Automated Algorithm Design by Integrating Modular Benchmarking Frameworks
• The Inefficiency of Genetic Programming for Symbolic Regression
• Algorithm Instance Footprint: Separating Easily Solvable and Challenging Problem Instances
• Tools for Landscape Analysis of Optimisation Problems in Procedural Content Generation for Games
• A Complementarity Analysis of the COCO Benchmark Problems and Artificially Generated Problems
• Cooperative Coevolution for Non-Separable Large-Scale Black-Box Optimization: Convergence Analyses and Distributed Accelerations
• Improving Nevergrad's Algorithm Selection Wizard NGOpt through Automated Algorithm Configuration
• Benchmarking Parameter Control Methods in Differential Evolution for Mixed-Integer Black-Box Optimization
• Towards Exploratory Landscape Analysis for Large-scale Optimization: A Dimensionality Reduction Framework
• Smart Starts: Accelerating Convergence through Uncommon Region Exploration
• Large Language Model as Meta-Surrogate for Data-Driven Many-Task Optimization: A Proof-of-Principle Study
• Surrogate-Based Black-Box Optimization Method for Costly Molecular Properties
• A Two-phase Framework with a Bézier Simplex-based Interpolation Method for Computationally Expensive Multi-objective Optimization
• Island-Based Evolutionary Computation with Diverse Surrogates and Adaptive Knowledge Transfer for High-Dimensional Data-Driven Optimization
• Comparison of High-Dimensional Bayesian Optimization Algorithms on BBOB
• Surrogate-guided optimization in quantum networks
• Temporal True and Surrogate Fitness Landscape Analysis for Expensive Bi-Objective Optimisation
• Does Dimensionality Reduction via Random Projections Preserve Landscape Features?
• SEvoBench : A C++ Framework For Evolutionary Single-Objective Optimization Benchmarking
• Hilbert curves for efficient exploratory landscape analysis neighbourhood sampling
• Optimization is Not Enough: Why Problem Formulation Deserves Equal Attention
• Benchmarking the Hooke-Jeeves Method, MTS-LS1, and BSrr on the Large-scale BBOB Function Set
• On Constructing Algorithm Portfolios in Algorithm Selection for Computationally Expensive Black-box Optimization in the Fixed-budget Setting