Perils of Label Indeterminacy: A Case Study on Prediction of Neurological Recovery After Cardiac Arrest

Abstract

The design of AI systems to assist human decision-making typically requires the availability of labels to train and evaluate supervised models. Frequently, however, these labels are unknown, and different ways of estimating them involve unverifiable assumptions or arbitrary choices. In this work, we introduce the concept of label indeterminacy and derive important implications in high-stakes AI-assisted decision-making. We present an empirical study in a healthcare context, focusing specifically on predicting the recovery of comatose patients after resuscitation from cardiac arrest. Our study shows that label indeterminacy can result in models that perform similarly when evaluated on patients with known labels, but vary drastically in their predictions for patients where labels are unknown. After demonstrating crucial ethical implications of label indeterminacy in this high-stakes context, we discuss takeaways for evaluation, reporting, and design.

The 2025 ACM Conference on Fairness, Accountability, and Transparency (FAccT '25)

References in corpus (9)

• To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-assisted Decision-making
• Expanding Explainability: Towards Social Transparency in AI systems
• Problem Formulation and Fairness
• Appropriate Reliance on AI Advice: Conceptualization and the Effect of Explanations
• Jury Learning: Integrating Dissenting Voices into Machine Learning Models
• Knowing About Knowing: An Illusion of Human Competence Can Hinder Appropriate Reliance on AI Systems
• Rethinking Human-AI Collaboration in Complex Medical Decision Making: A Case Study in Sepsis Diagnosis
• Explanations, Fairness, and Appropriate Reliance in Human-AI Decision-Making
• AI Reliance and Decision Quality: Fundamentals, Interdependence, and the Effects of Interventions