Multi-Objectivizing Software Configuration Tuning (for a single performance concern)

Abstract

Automatically tuning software configuration for optimizing a single performance attribute (e.g., minimizing latency) is not trivial, due to the nature of the configuration systems (e.g., complex landscape and expensive measurement). To deal with the problem, existing work has been focusing on developing various effective optimizers. However, a prominent issue that all these optimizers need to take care of is how to avoid the search being trapped in local optima -- a hard nut to crack for software configuration tuning due to its rugged and sparse landscape, and neighboring configurations tending to behave very differently. Overcoming such in an expensive measurement setting is even more challenging. In this paper, we take a different perspective to tackle this issue. Instead of focusing on improving the optimizer, we work on the level of optimization model. We do this by proposing a meta multi-objectivization model (MMO) that considers an auxiliary performance objective (e.g., throughput in addition to latency). What makes this model unique is that we do not optimize the auxiliary performance objective, but rather use it to make similarly-performing while different configurations less comparable (i.e. Pareto nondominated to each other), thus preventing the search from being trapped in local optima. Experiments on eight real-world software systems/environments with diverse performance attributes reveal that our MMO model is statistically more effective than state-of-the-art single-objective counterparts in overcoming local optima (up to 42% gain), while using as low as 24% of their measurements to achieve the same (or better) performance result.

arXiv comment13 pages, 7 figures, 4 tables. In Proceedings of the 29th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE'21), 2021

References in corpus (8)

• ParamILS: An Automatic Algorithm Configuration Framework
• BestConfig: Tapping the Performance Potential of Systems via Automatic Configuration Tuning
• How to Evaluate Solutions in Pareto-based Search-Based Software Engineering? A Critical Review and Methodological Guidance
• A Survey and Taxonomy of Self-Aware and Self-Adaptive Cloud Autoscaling Systems
• FEMOSAA: Feature Guided and Knee Driven Multi-Objective Optimization for Self-Adaptive Software
• Self-Adaptive Trade-off Decision Making for Autoscaling Cloud-Based Services
• DATESSO: Self-Adapting Service Composition with Debt-Aware Two Levels Constraint Reasoning
• Synergizing Domain Expertise with Self-Awareness in Software Systems: A Patternized Architecture Guideline

Cited by in corpus (8)

• The Weights can be Harmful: Pareto Search versus Weighted Search in Multi-Objective Search-Based Software Engineering
• Do Performance Aspirations Matter for Guiding Software Configuration Tuning?
• Does Configuration Encoding Matter in Learning Software Performance? An Empirical Study on Encoding Schemes
• Accuracy Can Lie: On the Impact of Surrogate Model in Configuration Tuning
• Adapting Multi-objectivized Software Configuration Tuning
• AI-driven Java Performance Testing: Balancing Result Quality with Testing Time
• Planning Landscape Analysis for Self-Adaptive Systems
• Test Case Minimization with Quantum Annealers