Better Software Analytics via "DUO": Data Mining Algorithms Using/Used-by Optimizers

Abstract

This paper claims that a new field of empirical software engineering research and practice is emerging: data mining using/used-by optimizers for empirical studies or DUO. For example, data miners can generate models that are explored by optimizers. Also, optimizers can advise how to best adjust the control parameters of a data miner. This combined approach acts like an agent leaning over the shoulder of an analyst that advises "ask this question next" or "ignore that problem, it is not relevant to your goals". Further, those agents can help us build "better" predictive models, where "better" can be either greater predictive accuracy or faster modeling time (which, in turn, enables the exploration of a wider range of options). We also caution that the era of papers that just use data miners is coming to an end. Results obtained from an unoptimized data miner can be quickly refuted, just by applying an optimizer to produce a different (and better performing) model. Our conclusion, hence, is that for software analytics it is possible, useful and necessary to combine data mining and optimization using DUO.

arXiv comment35 Pages, Accepted to EMSE Journal

References in corpus (17)

• Scikit-learn: Machine Learning in Python
• SMOTE: Synthetic Minority Over-sampling Technique
• What is Wrong with Topic Modeling? (and How to Fix it Using Search-based Software Engineering)
• Tuning for Software Analytics: is it Really Necessary?
• Is "Better Data" Better than "Better Data Miners"? (On the Benefits of Tuning SMOTE for Defect Prediction)
• Easy over Hard: A Case Study on Deep Learning
• Using Bad Learners to find Good Configurations
• Finding Better Active Learners for Faster Literature Reviews
• 500+ Times Faster Than Deep Learning (A Case Study Exploring Faster Methods for Text Mining StackOverflow)
• Why is Differential Evolution Better than Grid Search for Tuning Defect Predictors?
• Hyperparameter Optimization for Effort Estimation
• Beyond Evolutionary Algorithms for Search-based Software Engineering
• Data-Driven Search-based Software Engineering
• Is One Hyperparameter Optimizer Enough?
• Transfer Learning with Bellwethers to find Good Configurations
• An Improved Generic Bet-and-Run Strategy for Speeding Up Stochastic Local Search
• Building Better Quality Predictors Using "$\epsilon$-Dominance"

Cited by in corpus (6)

• Learning to Optimize: A Primer and A Benchmark
• Benchmarking in Optimization: Best Practice and Open Issues
• How to "DODGE" Complex Software Analytics?
• What makes a good Node.js package? Investigating Users, Contributors, and Runnability
• Model Review: A PROMISEing Opportunity
• Creative and Correct: Requesting Diverse Code Solutions from AI Foundation Models