Human-AI Collaboration in Data Science: Exploring Data Scientists' Perceptions of Automated AI

Abstract

The rapid advancement of artificial intelligence (AI) is changing our lives in many ways. One application domain is data science. New techniques in automating the creation of AI, known as AutoAI or AutoML, aim to automate the work practices of data scientists. AutoAI systems are capable of autonomously ingesting and pre-processing data, engineering new features, and creating and scoring models based on a target objectives (e.g. accuracy or run-time efficiency). Though not yet widely adopted, we are interested in understanding how AutoAI will impact the practice of data science. We conducted interviews with 20 data scientists who work at a large, multinational technology company and practice data science in various business settings. Our goal is to understand their current work practices and how these practices might change with AutoAI. Reactions were mixed: while informants expressed concerns about the trend of automating their jobs, they also strongly felt it was inevitable. Despite these concerns, they remained optimistic about their future job security due to a view that the future of data science work will be a collaboration between humans and AI systems, in which both automation and human expertise are indispensable.

References in corpus (6)

• Scikit-learn: Machine Learning in Python
• Neural Architecture Search: A Survey
• Trust in Data Science: Collaboration, Translation, and Accountability in Corporate Data Science Projects
• How Data Scientists Work Together With Domain Experts in Scientific Collaborations: To Find The Right Answer Or To Ask The Right Question?
• Data Vision: Learning to See Through Algorithmic Abstraction
• One button machine for automating feature engineering in relational databases

Cited by in corpus (46)

• Designing Creative AI Partners with COFI: A Framework for Modeling Interaction in Human-AI Co-Creative Systems
• Intelligent Computing: The Latest Advances, Challenges and Future
• The Programmer's Assistant: Conversational Interaction with a Large Language Model for Software Development
• Human Factors in Model Interpretability: Industry Practices, Challenges, and Needs
• "Help Me Help the AI": Understanding How Explainability Can Support Human-AI Interaction
• How AI Developers Overcome Communication Challenges in a Multidisciplinary Team: A Case Study
• Trust in AutoML: Exploring Information Needs for Establishing Trust in Automated Machine Learning Systems
• PromptCharm: Text-to-Image Generation through Multi-modal Prompting and Refinement
• Documentation Matters: Human-Centered AI System to Assist Data Science Code Documentation in Computational Notebooks
• "It Felt Like Having a Second Mind": Investigating Human-AI Co-creativity in Prewriting with Large Language Models
• Better Together? An Evaluation of AI-Supported Code Translation
• The Impact of Imperfect XAI on Human-AI Decision-Making
• AutoAIViz: Opening the Blackbox of Automated Artificial Intelligence with Conditional Parallel Coordinates
• Telling Stories from Computational Notebooks: AI-Assisted Presentation Slides Creation for Presenting Data Science Work
• Automating Data Science: Prospects and Challenges
• Machine Learning Practices Outside Big Tech: How Resource Constraints Challenge Responsible Development
• Increasing the Speed and Accuracy of Data LabelingThrough an AI Assisted Interface
• Path To Gain Functional Transparency In Artificial Intelligence With Meaningful Explainability
• Out of Context: Investigating the Bias and Fairness Concerns of "Artificial Intelligence as a Service"
• Eliciting Best Practices for Collaboration with Computational Notebooks
• AutoML in The Wild: Obstacles, Workarounds, and Expectations
• Model Sketching: Centering Concepts in Early-Stage Machine Learning Model Design
• Model Compression in Practice: Lessons Learned from Practitioners Creating On-device Machine Learning Experiences
• CoPrompt: Supporting Prompt Sharing and Referring in Collaborative Natural Language Programming
• Ground Truth Or Dare: Factors Affecting The Creation Of Medical Datasets For Training AI
• "We Have No Idea How Models will Behave in Production until Production": How Engineers Operationalize Machine Learning
• UXAgent: An LLM Agent-Based Usability Testing Framework for Web Design
• Why, How and Where of Delays in Software Security Patch Management: An Empirical Investigation in the Healthcare Sector
• Why is AI not a Panacea for Data Workers? An Interview Study on Human-AI Collaboration in Data Storytelling
• MetaAgents: Large Language Model Based Agents for Decision-Making on Teaming
• AutoDOViz: Human-Centered Automation for Decision Optimization
• Enabling Collaborative Data Science Development with the Ballet Framework
• Dirty Data in the Newsroom: Comparing Data Preparation in Journalism and Data Science
• Assessing the Use of AutoML for Data-Driven Software Engineering
• A Risk Taxonomy and Reflection Tool for Large Language Model Adoption in Public Health
• Interaction Configurations and Prompt Guidance in Conversational AI for Question Answering in Human-AI Teams
• PEANUT: A Human-AI Collaborative Tool for Annotating Audio-Visual Data
• "Is It My Turn?" Assessing Teamwork and Taskwork in Collaborative Immersive Analytics
• Understanding the Dataset Practitioners Behind Large Language Model Development
• Towards Feature Engineering with Human and AI's Knowledge: Understanding Data Science Practitioners' Perceptions in Human&AI-Assisted Feature Engineering Design
• BDIViz: An Interactive Visualization System for Biomedical Schema Matching with LLM-Powered Validation
• Understanding Collaboration between Professional Designers and Decision-making AI: A Case Study in the Workplace
• Observing Fine-Grained Changes in Jupyter Notebooks During Development Time
• Supporting Serendipity: Opportunities and Challenges for Human-AI Collaboration in Qualitative Analysis
• Customer Service Representative's Perception of the AI Assistant in an Organization's Call Center
• From Doyle to AGM: A Survey and an Implementation Roadmap for Belief Change