Can AI Model the Complexities of Human Moral Decision-Making? A Qualitative Study of Kidney Allocation Decisions

Abstract

A growing body of work in Ethical AI attempts to capture human moral judgments through simple computational models. The key question we address in this work is whether such simple AI models capture {the critical} nuances of moral decision-making by focusing on the use case of kidney allocation. We conducted twenty interviews where participants explained their rationale for their judgments about who should receive a kidney. We observe participants: (a) value patients' morally-relevant attributes to different degrees; (b) use diverse decision-making processes, citing heuristics to reduce decision complexity; (c) can change their opinions; (d) sometimes lack confidence in their decisions (e.g., due to incomplete information); and (e) express enthusiasm and concern regarding AI assisting humans in kidney allocation decisions. Based on these findings, we discuss challenges of computationally modeling moral judgments {as a stand-in for human input}, highlight drawbacks of current approaches, and suggest future directions to address these issues.

In ACM Conference on Human Factors in Computing Systems (CHI), 2025

References in corpus (10)

• Expanding Explainability: Towards Social Transparency in AI systems
• Power to the People? Opportunities and Challenges for Participatory AI
• Algorithmic decision-making in AVs: Understanding ethical and technical concerns for smart cities
• Beyond Expertise and Roles: A Framework to Characterize the Stakeholders of Interpretable Machine Learning and their Needs
• Adapting a Kidney Exchange Algorithm to Align with Human Values
• Collective Constitutional AI: Aligning a Language Model with Public Input
• Human Perceptions on Moral Responsibility of AI: A Case Study in AI-Assisted Bail Decision-Making
• The Conflict Between Explainable and Accountable Decision-Making Algorithms
• On Quantified Modal Theorem Proving for Modeling Ethics
• Developing moral AI to support antimicrobial decision making