How to "DODGE" Complex Software Analytics?

Abstract

Machine learning techniques applied to software engineering tasks can be improved by hyperparameter optimization, i.e., automatic tools that find good settings for a learner's control parameters. We show that such hyperparameter optimization can be unnecessarily slow, particularly when the optimizers waste time exploring "redundant tunings"', i.e., pairs of tunings which lead to indistinguishable results. By ignoring redundant tunings, DODGE, a tuning tool, runs orders of magnitude faster, while also generating learners with more accurate predictions than seen in prior state-of-the-art approaches.

13 Pages, Accepted to IEEE Transactions in Software Engineering, 2019

References in corpus (11)

• Scikit-learn: Machine Learning in Python
• SMOTE: Synthetic Minority Over-sampling Technique
• NLTK: The Natural Language Toolkit
• What is Wrong with Topic Modeling? (and How to Fix it Using Search-based Software Engineering)
• Tuning for Software Analytics: is it Really Necessary?
• Is "Better Data" Better than "Better Data Miners"? (On the Benefits of Tuning SMOTE for Defect Prediction)
• 500+ Times Faster Than Deep Learning (A Case Study Exploring Faster Methods for Text Mining StackOverflow)
• Why is Differential Evolution Better than Grid Search for Tuning Defect Predictors?
• Hyperparameter Optimization for Effort Estimation
• Better Software Analytics via "DUO": Data Mining Algorithms Using/Used-by Optimizers
• Is One Hyperparameter Optimizer Enough?

Cited by in corpus (14)

• Predicting Defective Lines Using a Model-Agnostic Technique
• Simpler Hyperparameter Optimization for Software Analytics: Why, How, When?
• Hyperparameter Optimization for AST Differencing
• Understanding the Automated Parameter Optimization on Transfer Learning for CPDP: An Empirical Study
• BiLO-CPDP: Bi-Level Programming for Automated Model Discovery in Cross-Project Defect Prediction
• Early Life Cycle Software Defect Prediction. Why? How?
• FRUGAL: Unlocking SSL for Software Analytics
• Better Data Labelling with EMBLEM (and how that Impacts Defect Prediction)
• Model Review: A PROMISEing Opportunity
• Learning to Recognize Actionable Static Code Warnings (is Intrinsically Easy)
• Building Very Small Test Suites (with Snap)
• Faster SAT Solving for Software with Repeated Structures (with Case Studies on Software Test Suite Minimization)
• How to Better Distinguish Security Bug Reports (using Dual Hyperparameter Optimization
• Mining Scientific Workflows for Anomalous Data Transfers