Multimodal Healthcare AI: Identifying and Designing Clinically Relevant Vision-Language Applications for Radiology
arXiv:2402.14252 · doi:10.1145/3613904.3642013
Abstract
Recent advances in AI combine large language models (LLMs) with vision encoders that bring forward unprecedented technical capabilities to leverage for a wide range of healthcare applications. Focusing on the domain of radiology, vision-language models (VLMs) achieve good performance results for tasks such as generating radiology findings based on a patient's medical image, or answering visual questions (e.g., 'Where are the nodules in this chest X-ray?'). However, the clinical utility of potential applications of these capabilities is currently underexplored. We engaged in an iterative, multidisciplinary design process to envision clinically relevant VLM interactions, and co-designed four VLM use concepts: Draft Report Generation, Augmented Report Review, Visual Search and Querying, and Patient Imaging History Highlights. We studied these concepts with 13 radiologists and clinicians who assessed the VLM concepts as valuable, yet articulated many design considerations. Reflecting on our findings, we discuss implications for integrating VLM capabilities in radiology, and for healthcare AI more generally.
to appear at CHI 2024
References in corpus (13)
- Improving fairness in machine learning systems: What do industry practitioners need?
- Unremarkable AI: Fitting Intelligent Decision Support into Critical, Clinical Decision-Making Processes
- Designing AI for Trust and Collaboration in Time-Constrained Medical Decisions: A Sociotechnical Lens
- How AI Developers Overcome Communication Challenges in a Multidisciplinary Team: A Case Study
- CheXplain: Enabling Physicians to Explore and UnderstandData-Driven, AI-Enabled Medical Imaging Analysis
- A Systematic Review and Thematic Analysis of Community-Collaborative Approaches to Computing Research
- Understanding Frontline Workers' and Unhoused Individuals' Perspectives on AI Used in Homeless Services
- Deliberating with AI: Improving Decision-Making for the Future through Participatory AI Design and Stakeholder Deliberation
- "If I Had All the Time in the World": Ophthalmologists' Perceptions of Anchoring Bias Mitigation in Clinical AI Support
- How Do UX Practitioners Communicate AI as a Design Material? Artifacts, Conceptions, and Propositions
- Augmenting Pathologists with NaviPath: Design and Evaluation of a Human-AI Collaborative Navigation System
- Sketching AI Concepts with Capabilities and Examples: AI Innovation in the Intensive Care Unit
- Investigating Why Clinicians Deviate from Standards of Care: Liberating Patients from Mechanical Ventilation in the ICU