Whither AutoML? Understanding the Role of Automation in Machine Learning Workflows

Abstract

Efforts to make machine learning more widely accessible have led to a rapid increase in Auto-ML tools that aim to automate the process of training and deploying machine learning. To understand how Auto-ML tools are used in practice today, we performed a qualitative study with participants ranging from novice hobbyists to industry researchers who use Auto-ML tools. We present insights into the benefits and deficiencies of existing tools, as well as the respective roles of the human and automation in ML workflows. Finally, we discuss design implications for the future of Auto-ML tool development. We argue that instead of full automation being the ultimate goal of Auto-ML, designers of these tools should focus on supporting a partnership between the user and the Auto-ML tool. This means that a range of Auto-ML tools will need to be developed to support varying user goals such as simplicity, reproducibility, and reliability.

References in corpus (6)

• The NumPy array: a structure for efficient numerical computation
• Snorkel: Rapid Training Data Creation with Weak Supervision
• Human Factors in Model Interpretability: Industry Practices, Challenges, and Needs
• AutoAIViz: Opening the Blackbox of Automated Artificial Intelligence with Conditional Parallel Coordinates
• Demystifying a Dark Art: Understanding Real-World Machine Learning Model Development
• PipelineProfiler: A Visual Analytics Tool for the Exploration of AutoML Pipelines

Cited by in corpus (2)

• Can Fairness be Automated? Guidelines and Opportunities for Fairness-aware AutoML
• Human-Centered AI for Data Science: A Systematic Approach