Bridging the gap: Towards an Expanded Toolkit for AI-driven Decision-Making in the Public Sector

Abstract

AI-driven decision-making systems are becoming instrumental in the public sector, with applications spanning areas like criminal justice, social welfare, financial fraud detection, and public health. While these systems offer great potential benefits to institutional decision-making processes, such as improved efficiency and reliability, these systems face the challenge of aligning machine learning (ML) models with the complex realities of public sector decision-making. In this paper, we examine five key challenges where misalignment can occur, including distribution shifts, label bias, the influence of past decision-making on the data side, as well as competing objectives and human-in-the-loop on the model output side. Our findings suggest that standard ML methods often rely on assumptions that do not fully account for these complexities, potentially leading to unreliable and harmful predictions. To address this, we propose a shift in modeling efforts from focusing solely on predictive accuracy to improving decision-making outcomes. We offer guidance for selecting appropriate modeling frameworks, including counterfactual prediction and policy learning, by considering how the model estimand connects to the decision-maker's utility. Additionally, we outline technical methods that address specific challenges within each modeling approach. Finally, we argue for the importance of external input from domain experts and stakeholders to ensure that model assumptions and design choices align with real-world policy objectives, taking a step towards harmonizing AI and public sector objectives.

References in corpus (36)

• Towards A Rigorous Science of Interpretable Machine Learning
• Interpretable machine learning: definitions, methods, and applications
• Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods
• Domain Generalization: A Survey
• Meta-learners for Estimating Heterogeneous Treatment Effects using Machine Learning
• Prediction-Based Decisions and Fairness: A Catalogue of Choices, Assumptions, and Definitions
• Characterizing Concept Drift
• Performance guarantees for individualized treatment rules
• A Survey of Multi-Objective Sequential Decision-Making
• Doubly Robust Policy Evaluation and Learning
• An introduction to domain adaptation and transfer learning
• Fairness in Credit Scoring: Assessment, Implementation and Profit Implications
• Statistical inference for the mean outcome under a possibly non-unique optimal treatment strategy
• Representation Bias in Data: A Survey on Identification and Resolution Techniques
• Algorithms and Decision-Making in the Public Sector
• Explainable Machine Learning for Public Policy: Use Cases, Gaps, and Research Directions
• Supervised Domain Adaptation: A Graph Embedding Perspective and a Rectified Experimental Protocol
• Difficult Lessons on Social Prediction from Wisconsin Public Schools
• Data-Driven Allocation of Preventive Care With Application to Diabetes Mellitus Type II
• A Unifying Causal Framework for Analyzing Dataset Shift-stable Learning Algorithms
• Sources of Uncertainty in Supervised Machine Learning -- A Statisticians' View
• Multi-Objective Automatic Machine Learning with AutoxgboostMC
• On the Distinction Between "Conditional Average Treatment Effects" (CATE) and "Individual Treatment Effects" (ITE) Under Ignorability Assumptions
• Machine Learning Who to Nudge: Causal vs Predictive Targeting in a Field Experiment on Student Financial Aid Renewal
• Treatment Allocation under Uncertain Costs
• Prescriptive Machine Learning for Automated Decision Making: Challenges and Opportunities
• Empirical Welfare Maximization with Constraints
• Flexible machine learning estimation of conditional average treatment effects: a blessing and a curse
• Uncertainty-aware predictive modeling for fair data-driven decisions
• A Justice-Based Framework for the Analysis of Algorithmic Fairness-Utility Trade-Offs
• Multi-CATE: Multi-Accurate Conditional Average Treatment Effect Estimation Robust to Unknown Covariate Shifts
• Causal Ordering Without Effect Estimation: A Framework for Using Proxies in Treatment Prioritization
• Policy Learning with Rare Outcomes
• Fair Off-Policy Learning from Observational Data
• Learning with Selectively Labeled Data from Multiple Decision-makers
• The Relative Value of Prediction in Algorithmic Decision Making

Cited by in corpus (1)

• Human-AI Interaction Requirements in Public Sector Procurements