AutoAIViz: Opening the Blackbox of Automated Artificial Intelligence with Conditional Parallel Coordinates

Abstract

Artificial Intelligence (AI) can now automate the algorithm selection, feature engineering, and hyperparameter tuning steps in a machine learning workflow. Commonly known as AutoML or AutoAI, these technologies aim to relieve data scientists from the tedious manual work. However, today's AutoAI systems often present only limited to no information about the process of how they select and generate model results. Thus, users often do not understand the process, neither do they trust the outputs. In this short paper, we provide a first user evaluation by 10 data scientists of an experimental system, AutoAIViz, that aims to visualize AutoAI's model generation process. We find that the proposed system helps users to complete the data science tasks, and increases their understanding, toward the goal of increasing trust in the AutoAI system.

5 pages, 1 figure, IUI2020

References in corpus (3)

• Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Making
• Trust in Data Science: Collaboration, Translation, and Accountability in Corporate Data Science Projects
• One button machine for automating feature engineering in relational databases

Cited by in corpus (14)

• "Brilliant AI Doctor" in Rural China: Tensions and Challenges in AI-Powered CDSS Deployment
• Documentation Matters: Human-Centered AI System to Assist Data Science Code Documentation in Computational Notebooks
• AutoDS: Towards Human-Centered Automation of Data Science
• Out of Context: Investigating the Bias and Fairness Concerns of "Artificial Intelligence as a Service"
• AutoML in The Wild: Obstacles, Workarounds, and Expectations
• How Much Automation Does a Data Scientist Want?
• Model LineUpper: Supporting Interactive Model Comparison at Multiple Levels for AutoML
• Why is AI not a Panacea for Data Workers? An Interview Study on Human-AI Collaboration in Data Storytelling
• Compress and Compare: Interactively Evaluating Efficiency and Behavior Across ML Model Compression Experiments
• AutoDOViz: Human-Centered Automation for Decision Optimization
• Preserve, Promote, or Attack? GNN Explanation via Topology Perturbation
• PipelineProfiler: A Visual Analytics Tool for the Exploration of AutoML Pipelines
• Whither AutoML? Understanding the Role of Automation in Machine Learning Workflows
• Assessment of cognitive characteristics in intelligent systems and predictive ability