Trust in Data Science: Collaboration, Translation, and Accountability in Corporate Data Science Projects

Abstract

The trustworthiness of data science systems in applied and real-world settings emerges from the resolution of specific tensions through situated, pragmatic, and ongoing forms of work. Drawing on research in CSCW, critical data studies, and history and sociology of science, and six months of immersive ethnographic fieldwork with a corporate data science team, we describe four common tensions in applied data science work: (un)equivocal numbers, (counter)intuitive knowledge, (in)credible data, and (in)scrutable models. We show how organizational actors establish and re-negotiate trust under messy and uncertain analytic conditions through practices of skepticism, assessment, and credibility. Highlighting the collaborative and heterogeneous nature of real-world data science, we show how the management of trust in applied corporate data science settings depends not only on pre-processing and quantification, but also on negotiation and translation. We conclude by discussing the implications of our findings for data science research and practice, both within and beyond CSCW.

References in corpus (5)

• Model-Agnostic Interpretability of Machine Learning
• 'It's Reducing a Human Being to a Percentage'; Perceptions of Justice in Algorithmic Decisions
• Data Vision: Learning to See Through Algorithmic Abstraction
• Programs as Black-Box Explanations
• "From What I see, this makes sense": Seeing meaning in algorithmic results

Cited by in corpus (31)

• Human-AI Collaboration in Data Science: Exploring Data Scientists' Perceptions of Automated AI
• The Fallacy of AI Functionality
• Problem Formulation and Fairness
• How AI Developers Overcome Communication Challenges in a Multidisciplinary Team: A Case Study
• Trust in AutoML: Exploring Information Needs for Establishing Trust in Automated Machine Learning Systems
• Seeing Like a Toolkit: How Toolkits Envision the Work of AI Ethics
• This Thing Called Fairness: Disciplinary Confusion Realizing a Value in Technology
• Exploring How Machine Learning Practitioners (Try To) Use Fairness Toolkits
• The Who in XAI: How AI Background Shapes Perceptions of AI Explanations
• Sensible AI: Re-imagining Interpretability and Explainability using Sensemaking Theory
• Generative AI in the Wild: Prospects, Challenges, and Strategies
• AutoAIViz: Opening the Blackbox of Automated Artificial Intelligence with Conditional Parallel Coordinates
• Facilitating Knowledge Sharing from Domain Experts to Data Scientists for Building NLP Models
• Seamful XAI: Operationalizing Seamful Design in Explainable AI
• How Do UX Practitioners Communicate AI as a Design Material? Artifacts, Conceptions, and Propositions
• Knowledge Graphs in Practice: Characterizing their Users, Challenges, and Visualization Opportunities
• Eliciting Best Practices for Collaboration with Computational Notebooks
• AutoML in The Wild: Obstacles, Workarounds, and Expectations
• Model Sketching: Centering Concepts in Early-Stage Machine Learning Model Design
• Model Compression in Practice: Lessons Learned from Practitioners Creating On-device Machine Learning Experiences
• An Exploratory Study on Visual Exploration of Model Simulations by Multiple Types of Experts
• "We Have No Idea How Models will Behave in Production until Production": How Engineers Operationalize Machine Learning
• Situating Data Sets: Making Public Data Actionable for Housing Justice
• Are Metrics Enough? Guidelines for Communicating and Visualizing Predictive Models to Subject Matter Experts
• The Cadaver in the Machine: The Social Practices of Measurement and Validation in Motion Capture Technology
• Thoughtful Adoption of NLP for Civic Participation: Understanding Differences Among Policymakers
• Supporting Industry Computing Researchers in Assessing, Articulating, and Addressing the Potential Negative Societal Impact of Their Work
• Towards Creating Infrastructures for Values and Ethics Work in the Production of Software Technologies
• Towards Transparent, Reusable, and Customizable Data Science in Computational Notebooks
• Tempo: Helping Data Scientists and Domain Experts Collaboratively Specify Predictive Modeling Tasks
• Outlining Traceability: A Principle for Operationalizing Accountability in Computing Systems