Recent Trends in the Use of Statistical Tests for Comparing Swarm and Evolutionary Computing Algorithms: Practical Guidelines and a Critical Review

Abstract

A key aspect of the design of evolutionary and swarm intelligence algorithms is studying their performance. Statistical comparisons are also a crucial part which allows for reliable conclusions to be drawn. In the present paper we gather and examine the approaches taken from different perspectives to summarise the assumptions made by these statistical tests, the conclusions reached and the steps followed to perform them correctly. In this paper, we conduct a survey on the current trends of the proposals of statistical analyses for the comparison of algorithms of computational intelligence and include a description of the statistical background of these tests. We illustrate the use of the most common tests in the context of the Competition on single-objective real parameter optimisation of the IEEE Congress on Evolutionary Computation (CEC) 2017 and describe the main advantages and drawbacks of the use of each kind of test and put forward some recommendations concerning their use.

arXiv comment52 pages, 10 figures, 19 tables

References in corpus (2)

• Benchmarking Evolutionary Algorithms For Single Objective Real-valued Constrained Optimization - A Critical Review
• Sample size estimation for power and accuracy in the experimental comparison of algorithms

Cited by in corpus (10)

• Random vector functional link network: recent developments, applications, and future directions
• A Prescription of Methodological Guidelines for Comparing Bio-inspired Optimization Algorithms
• How to avoid machine learning pitfalls: a guide for academic researchers
• Anomaly Detection in Predictive Maintenance: A New Evaluation Framework for Temporal Unsupervised Anomaly Detection Algorithms
• Simulated annealing for optimization of graphs and sequences
• Upgrading edges in the maximal covering location problem
• A bi-level encoding scheme for the clustered shortest-path tree problem in multifactorial optimization
• Scalable Feature Subset Selection for Big Data using Parallel Hybrid Evolutionary Algorithm based Wrapper in Apache Spark
• A Metaheuristic Approach for Mining Gradual Patterns
• HRA: A Multi-Criteria Framework for Ranking Metaheuristic Optimization Algorithms