Bridging the gap: Towards an Expanded Toolkit for AI-driven Decision-Making in the Public Sector
arXiv:2310.19091 · doi:10.1016/j.giq.2024.101976
Abstract
AI-driven decision-making systems are becoming instrumental in the public sector, with applications spanning areas like criminal justice, social welfare, financial fraud detection, and public health. While these systems offer great potential benefits to institutional decision-making processes, such as improved efficiency and reliability, these systems face the challenge of aligning machine learning (ML) models with the complex realities of public sector decision-making. In this paper, we examine five key challenges where misalignment can occur, including distribution shifts, label bias, the influence of past decision-making on the data side, as well as competing objectives and human-in-the-loop on the model output side. Our findings suggest that standard ML methods often rely on assumptions that do not fully account for these complexities, potentially leading to unreliable and harmful predictions. To address this, we propose a shift in modeling efforts from focusing solely on predictive accuracy to improving decision-making outcomes. We offer guidance for selecting appropriate modeling frameworks, including counterfactual prediction and policy learning, by considering how the model estimand connects to the decision-maker's utility. Additionally, we outline technical methods that address specific challenges within each modeling approach. Finally, we argue for the importance of external input from domain experts and stakeholders to ensure that model assumptions and design choices align with real-world policy objectives, taking a step towards harmonizing AI and public sector objectives.
References in corpus (13)
- Towards A Rigorous Science of Interpretable Machine Learning
- Performance guarantees for individualized treatment rules
- Doubly Robust Policy Evaluation and Learning
- An introduction to domain adaptation and transfer learning
- Algorithms and Decision-Making in the Public Sector
- Data-Driven Allocation of Preventive Care With Application to Diabetes Mellitus Type II
- On the Distinction Between "Conditional Average Treatment Effects" (CATE) and "Individual Treatment Effects" (ITE) Under Ignorability Assumptions
- Machine Learning Who to Nudge: Causal vs Predictive Targeting in a Field Experiment on Student Financial Aid Renewal
- Prescriptive Machine Learning for Automated Decision Making: Challenges and Opportunities
- Uncertainty-aware predictive modeling for fair data-driven decisions
- Learning with Selectively Labeled Data from Multiple Decision-makers
- The Relative Value of Prediction in Algorithmic Decision Making
- Multi-CATE: Multi-Accurate Conditional Average Treatment Effect Estimation Robust to Unknown Covariate Shifts