AutoDS: Towards Human-Centered Automation of Data Science

Abstract

Data science (DS) projects often follow a lifecycle that consists of laborious tasks for data scientists and domain experts (e.g., data exploration, model training, etc.). Only till recently, machine learning(ML) researchers have developed promising automation techniques to aid data workers in these tasks. This paper introduces AutoDS, an automated machine learning (AutoML) system that aims to leverage the latest ML automation techniques to support data science projects. Data workers only need to upload their dataset, then the system can automatically suggest ML configurations, preprocess data, select algorithm, and train the model. These suggestions are presented to the user via a web-based graphical user interface and a notebook-based programming user interface. We studied AutoDS with 30 professional data scientists, where one group used AutoDS, and the other did not, to complete a data science project. As expected, AutoDS improves productivity; Yet surprisingly, we find that the models produced by the AutoDS group have higher quality and less errors, but lower human confidence scores. We reflect on the findings by presenting design implications for incorporating automation techniques into human work in the data science lifecycle.

References in corpus (8)

• Metrics for Explainable AI: Challenges and Prospects
• "Brilliant AI Doctor" in Rural China: Tensions and Challenges in AI-Powered CDSS Deployment
• How AI Developers Overcome Communication Challenges in a Multidisciplinary Team: A Case Study
• Trust in AutoML: Exploring Information Needs for Establishing Trust in Automated Machine Learning Systems
• Calendar.help: Designing a Workflow-Based Scheduling Agent with Humans in the Loop
• One button machine for automating feature engineering in relational databases
• AutoAIViz: Opening the Blackbox of Automated Artificial Intelligence with Conditional Parallel Coordinates
• How Much Automation Does a Data Scientist Want?

Cited by in corpus (6)

• "Brilliant AI Doctor" in Rural China: Tensions and Challenges in AI-Powered CDSS Deployment
• How AI Developers Overcome Communication Challenges in a Multidisciplinary Team: A Case Study
• CASS: Towards Building a Social-Support Chatbot for Online Health Community
• Model LineUpper: Supporting Interactive Model Comparison at Multiple Levels for AutoML
• Human-Centered AI for Data Science: A Systematic Approach
• Naive Automated Machine Learning -- A Late Baseline for AutoML