HyperTendril: Visual Analytics for User-Driven Hyperparameter Optimization of Deep Neural Networks
arXiv:2009.02078 · doi:10.1109/TVCG.2020.3030380
Abstract
To mitigate the pain of manually tuning hyperparameters of deep neural networks, automated machine learning (AutoML) methods have been developed to search for an optimal set of hyperparameters in large combinatorial search spaces. However, the search results of AutoML methods significantly depend on initial configurations, making it a non-trivial task to find a proper configuration. Therefore, human intervention via a visual analytic approach bears huge potential in this task. In response, we propose HyperTendril, a web-based visual analytics system that supports user-driven hyperparameter tuning processes in a model-agnostic environment. HyperTendril takes a novel approach to effectively steering hyperparameter optimization through an iterative, interactive tuning procedure that allows users to refine the search spaces and the configuration of the AutoML method based on their own insights from given results. Using HyperTendril, users can obtain insights into the complex behaviors of various hyperparameter search algorithms and diagnose their configurations. In addition, HyperTendril supports variable importance analysis to help the users refine their search spaces based on the analysis of relative importance of different hyperparameters and their interaction effects. We present the evaluation demonstrating how HyperTendril helps users steer their tuning processes via a longitudinal user study based on the analysis of interaction logs and in-depth interviews while we deploy our system in a professional industrial environment.
arXiv commentWill be presented at IEEE VAST 2020 and published in IEEE Transactions on Visualization and Computer Graphics
References in corpus (16)
- Adam: A Method for Stochastic Optimization
- Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization
- Tune: A Research Platform for Distributed Model Selection and Training
- Tunability: Importance of Hyperparameters of Machine Learning Algorithms
- explAIner: A Visual Analytics Framework for Interactive and Explainable Machine Learning
- Ray: A Distributed Framework for Emerging AI Applications
- Population Based Training of Neural Networks
- Hyperparameter Importance Across Datasets
- The State of the Art in Integrating Machine Learning into Visual Analytics
- NSML: Meet the MLaaS platform with a real-world case study
- ATMSeer: Increasing Transparency and Controllability in Automated Machine Learning
- Direct-Manipulation Visualization of Deep Networks
- NSML: A Machine Learning Platform That Enables You to Focus on Your Models
- AutoAIViz: Opening the Blackbox of Automated Artificial Intelligence with Conditional Parallel Coordinates
- CHOPT : Automated Hyperparameter Optimization Framework for Cloud-Based Machine Learning Platforms
- Auptimizer -- an Extensible, Open-Source Framework for Hyperparameter Tuning