ATMSeer: Increasing Transparency and Controllability in Automated Machine Learning

Abstract

To relieve the pain of manually selecting machine learning algorithms and tuning hyperparameters, automated machine learning (AutoML) methods have been developed to automatically search for good models. Due to the huge model search space, it is impossible to try all models. Users tend to distrust automatic results and increase the search budget as much as they can, thereby undermining the efficiency of AutoML. To address these issues, we design and implement ATMSeer, an interactive visualization tool that supports users in refining the search space of AutoML and analyzing the results. To guide the design of ATMSeer, we derive a workflow of using AutoML based on interviews with machine learning experts. A multi-granularity visualization is proposed to enable users to monitor the AutoML process, analyze the searched models, and refine the search space in real time. We demonstrate the utility and usability of ATMSeer through two case studies, expert interviews, and a user study with 13 end users.

arXiv commentPublished in the ACM Conference on Human Factors in Computing Systems (CHI), 2019, Glasgow, Scotland UK

References in corpus (1)

• Manifold: A Model-Agnostic Framework for Interpretation and Diagnosis of Machine Learning Models

Cited by in corpus (16)

• The State of the Art in Enhancing Trust in Machine Learning Models with the Use of Visualizations
• StackGenVis: Alignment of Data, Algorithms, and Models for Stacking Ensemble Learning Using Performance Metrics
• Trust in AutoML: Exploring Information Needs for Establishing Trust in Automated Machine Learning Systems
• AutoDS: Towards Human-Centered Automation of Data Science
• AutoAIViz: Opening the Blackbox of Automated Artificial Intelligence with Conditional Parallel Coordinates
• Visual Analysis of Discrimination in Machine Learning
• Out of Context: Investigating the Bias and Fairness Concerns of "Artificial Intelligence as a Service"
• HyperTendril: Visual Analytics for User-Driven Hyperparameter Optimization of Deep Neural Networks
• AutoML in The Wild: Obstacles, Workarounds, and Expectations
• VisEvol: Visual Analytics to Support Hyperparameter Search through Evolutionary Optimization
• DFSeer: A Visual Analytics Approach to Facilitate Model Selection for Demand Forecasting
• Characterizing Student Engagement Moods for Dropout Prediction in Question Pool Websites
• Towards Dataset-scale and Feature-oriented Evaluation of Text Summarization in Large Language Model Prompts
• AutoDOViz: Human-Centered Automation for Decision Optimization
• Enabling Collaborative Data Science Development with the Ballet Framework
• Practitioner Motives to Use Different Hyperparameter Optimization Methods