VisEvol: Visual Analytics to Support Hyperparameter Search through Evolutionary Optimization

Abstract

During the training phase of machine learning (ML) models, it is usually necessary to configure several hyperparameters. This process is computationally intensive and requires an extensive search to infer the best hyperparameter set for the given problem. The challenge is exacerbated by the fact that most ML models are complex internally, and training involves trial-and-error processes that could remarkably affect the predictive result. Moreover, each hyperparameter of an ML algorithm is potentially intertwined with the others, and changing it might result in unforeseeable impacts on the remaining hyperparameters. Evolutionary optimization is a promising method to try and address those issues. According to this method, performant models are stored, while the remainder are improved through crossover and mutation processes inspired by genetic algorithms. We present VisEvol, a visual analytics tool that supports interactive exploration of hyperparameters and intervention in this evolutionary procedure. In summary, our proposed tool helps the user to generate new models through evolution and eventually explore powerful hyperparameter combinations in diverse regions of the extensive hyperparameter space. The outcome is a voting ensemble (with equal rights) that boosts the final predictive performance. The utility and applicability of VisEvol are demonstrated with two use cases and interviews with ML experts who evaluated the effectiveness of the tool.

This manuscript is accepted for publication in a special issue of Computer Graphics Forum (CGF)

References in corpus (24)

• XGBoost: A Scalable Tree Boosting System
• UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction
• Practical Bayesian Optimization of Machine Learning Algorithms
• Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation
• Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization
• ParamILS: An Automatic Algorithm Configuration Framework
• Tune: A Research Platform for Distributed Model Selection and Training
• Tunability: Importance of Hyperparameters of Machine Learning Algorithms
• Hyperparameter Search in Machine Learning
• Manifold: A Model-Agnostic Framework for Interpretation and Diagnosis of Machine Learning Models
• Hyperparameter Importance Across Datasets
• The State of the Art in Enhancing Trust in Machine Learning Models with the Use of Visualizations
• t-viSNE: Interactive Assessment and Interpretation of t-SNE Projections
• StackGenVis: Alignment of Data, Algorithms, and Models for Stacking Ensemble Learning Using Performance Metrics
• Explainable Matrix -- Visualization for Global and Local Interpretability of Random Forest Classification Ensembles
• ATMSeer: Increasing Transparency and Controllability in Automated Machine Learning
• Autotune: A Derivative-free Optimization Framework for Hyperparameter Tuning
• Explaining Vulnerabilities to Adversarial Machine Learning through Visual Analytics
• NSML: A Machine Learning Platform That Enables You to Focus on Your Models
• AutoAIViz: Opening the Blackbox of Automated Artificial Intelligence with Conditional Parallel Coordinates
• Easy Hyperparameter Search Using Optunity
• HyperTendril: Visual Analytics for User-Driven Hyperparameter Optimization of Deep Neural Networks
• Progressive Data Science: Potential and Challenges
• CHOPT : Automated Hyperparameter Optimization Framework for Cloud-Based Machine Learning Platforms

Cited by in corpus (5)

• FeatureEnVi: Visual Analytics for Feature Engineering Using Stepwise Selection and Semi-Automatic Extraction Approaches
• VisRuler: Visual Analytics for Extracting Decision Rules from Bagged and Boosted Decision Trees
• HardVis: Visual Analytics to Handle Instance Hardness Using Undersampling and Oversampling Techniques
• Visual Analytics for Explainable and Trustworthy Artificial Intelligence
• Towards the Visualization of Aggregated Class Activation Maps to Analyse the Global Contribution of Class Features