Rethinking Human-AI Collaboration in Complex Medical Decision Making: A Case Study in Sepsis Diagnosis

Abstract

Today's AI systems for medical decision support often succeed on benchmark datasets in research papers but fail in real-world deployment. This work focuses on the decision making of sepsis, an acute life-threatening systematic infection that requires an early diagnosis with high uncertainty from the clinician. Our aim is to explore the design requirements for AI systems that can support clinical experts in making better decisions for the early diagnosis of sepsis. The study begins with a formative study investigating why clinical experts abandon an existing AI-powered Sepsis predictive module in their electrical health record (EHR) system. We argue that a human-centered AI system needs to support human experts in the intermediate stages of a medical decision-making process (e.g., generating hypotheses or gathering data), instead of focusing only on the final decision. Therefore, we build SepsisLab based on a state-of-the-art AI algorithm and extend it to predict the future projection of sepsis development, visualize the prediction uncertainty, and propose actionable suggestions (i.e., which additional laboratory tests can be collected) to reduce such uncertainty. Through heuristic evaluation with six clinicians using our prototype system, we demonstrate that SepsisLab enables a promising human-AI collaboration paradigm for the future of AI-assisted sepsis diagnosis and other high-stakes medical decision making.

Accepted by CHI'24

References in corpus (7)

• Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Making
• Dipole: Diagnosis Prediction in Healthcare via Attention-based Bidirectional Recurrent Neural Networks
• Unremarkable AI: Fitting Intelligent Decision Support into Critical, Clinical Decision-Making Processes
• "Brilliant AI Doctor" in Rural China: Tensions and Challenges in AI-Powered CDSS Deployment
• Designing AI for Trust and Collaboration in Time-Constrained Medical Decisions: A Sociotechnical Lens
• SepsisLab: Early Sepsis Prediction with Uncertainty Quantification and Active Sensing
• Contextual embedding and model weighting by fusing domain knowledge on Biomedical Question Answering

Cited by in corpus (5)

• AI, Help Me Think$\unicode{x2014}$but for Myself: Assisting People in Complex Decision-Making by Providing Different Kinds of Cognitive Support
• Beyond Recommendations: From Backward to Forward AI Support of Pilots' Decision-Making Process
• Vital Insight: Assisting Experts' Context-Driven Sensemaking of Multi-modal Personal Tracking Data Using Visualization and Human-In-The-Loop LLM
• Designing Human-AI System for Legal Research: A Case Study of Precedent Search in Chinese Law
• Perils of Label Indeterminacy: A Case Study on Prediction of Neurological Recovery After Cardiac Arrest