Problem Formulation and Fairness

Abstract

Formulating data science problems is an uncertain and difficult process. It requires various forms of discretionary work to translate high-level objectives or strategic goals into tractable problems, necessitating, among other things, the identification of appropriate target variables and proxies. While these choices are rarely self-evident, normative assessments of data science projects often take them for granted, even though different translations can raise profoundly different ethical concerns. Whether we consider a data science project fair often has as much to do with the formulation of the problem as any property of the resulting model. Building on six months of ethnographic fieldwork with a corporate data science team---and channeling ideas from sociology and history of science, critical data studies, and early writing on knowledge discovery in databases---we describe the complex set of actors and activities involved in problem formulation. Our research demonstrates that the specification and operationalization of the problem are always negotiated and elastic, and rarely worked out with explicit normative considerations in mind. In so doing, we show that careful accounts of everyday data science work can help us better understand how and why data science problems are posed in certain ways---and why specific formulations prevail in practice, even in the face of what might seem like normatively preferable alternatives. We conclude by discussing the implications of our findings, arguing that effective normative interventions will require attending to the practical work of problem formulation.

Conference on Fairness, Accountability, and Transparency (FAT* '19), January 29-31, 2019, Atlanta, GA, USA

References in corpus (2)

• Trust in Data Science: Collaboration, Translation, and Accountability in Corporate Data Science Projects
• Data Vision: Learning to See Through Algorithmic Abstraction

Cited by in corpus (40)

• Mitigating Bias in Algorithmic Hiring: Evaluating Claims and Practices
• Prediction-Based Decisions and Fairness: A Catalogue of Choices, Assumptions, and Definitions
• The Fallacy of AI Functionality
• Measurement and Fairness
• Roles for Computing in Social Change
• Does "AI" stand for augmenting inequality in the era of covid-19 healthcare?
• Accountability in an Algorithmic Society: Relationality, Responsibility, and Robustness in Machine Learning
• Seeing Like a Toolkit: How Toolkits Envision the Work of AI Ethics
• Algorithms and Decision-Making in the Public Sector
• Investigating How Practitioners Use Human-AI Guidelines: A Case Study on the People + AI Guidebook
• A Toolbox for Surfacing Health Equity Harms and Biases in Large Language Models
• Exploring How Machine Learning Practitioners (Try To) Use Fairness Toolkits
• Understanding accountability in algorithmic supply chains
• The Who in XAI: How AI Background Shapes Perceptions of AI Explanations
• Escaping the Impossibility of Fairness: From Formal to Substantive Algorithmic Fairness
• Deliberating with AI: Improving Decision-Making for the Future through Participatory AI Design and Stakeholder Deliberation
• `It is currently hodgepodge'': Examining AI/ML Practitioners' Challenges during Co-production of Responsible AI Values
• The Situate AI Guidebook: Co-Designing a Toolkit to Support Multi-Stakeholder Early-stage Deliberations Around Public Sector AI Proposals
• Awareness in Practice: Tensions in Access to Sensitive Attribute Data for Antidiscrimination
• Emergent Unfairness in Algorithmic Fairness-Accuracy Trade-Off Research
• Seamful XAI: Operationalizing Seamful Design in Explainable AI
• Sketching AI Concepts with Capabilities and Examples: AI Innovation in the Intensive Care Unit
• Model Sketching: Centering Concepts in Early-Stage Machine Learning Model Design
• Are We Asking the Right Questions?: Designing for Community Stakeholders' Interactions with AI in Policing
• Toward Operationalizing Pipeline-aware ML Fairness: A Research Agenda for Developing Practical Guidelines and Tools
• Multi-Target Multiplicity: Flexibility and Fairness in Target Specification under Resource Constraints
• Exploring AI Problem Formulation with Children via Teachable Machines
• The "Who", "What", and "How" of Responsible AI Governance: A Systematic Review and Meta-Analysis of (Actor, Stage)-Specific Tools
• Machine Learning Data Practices through a Data Curation Lens: An Evaluation Framework
• Flipping the Script on Criminal Justice Risk Assessment: An actuarial model for assessing the risk the federal sentencing system poses to defendants
• Supporting Industry Computing Researchers in Assessing, Articulating, and Addressing the Potential Negative Societal Impact of Their Work
• Making the Right Thing: Bridging HCI and Responsible AI in Early-Stage AI Concept Selection
• Nteasee: Understanding Needs in AI for Health in Africa -- A Mixed-Methods Study of Expert and General Population Perspectives
• Adversarial Scrutiny of Evidentiary Statistical Software
• Tempo: Helping Data Scientists and Domain Experts Collaboratively Specify Predictive Modeling Tasks
• Perils of Label Indeterminacy: A Case Study on Prediction of Neurological Recovery After Cardiac Arrest
• To LLM, or Not to LLM: How Designers and Developers Navigate LLMs as Tools or Teammates
• Falsifying Discriminant Validity of Predictive Algorithms
• A Bridge to Nowhere: A Healthcare Case Study for Non-Reformist Design
• Outlining Traceability: A Principle for Operationalizing Accountability in Computing Systems