"Brilliant AI Doctor" in Rural China: Tensions and Challenges in AI-Powered CDSS Deployment
arXiv:2101.01524 · doi:10.1145/3411764.3445432
Abstract
Artificial intelligence (AI) technology has been increasingly used in the implementation of advanced Clinical Decision Support Systems (CDSS). Research demonstrated the potential usefulness of AI-powered CDSS (AI-CDSS) in clinical decision making scenarios. However, post-adoption user perception and experience remain understudied, especially in developing countries. Through observations and interviews with 22 clinicians from 6 rural clinics in China, this paper reports the various tensions between the design of an AI-CDSS system ("Brilliant Doctor") and the rural clinical context, such as the misalignment with local context and workflow, the technical limitations and usability barriers, as well as issues related to transparency and trustworthiness of AI-CDSS. Despite these tensions, all participants expressed positive attitudes toward the future of AI-CDSS, especially acting as "a doctor's AI assistant" to realize a Human-AI Collaboration future in clinical settings. Finally we draw on our findings to discuss implications for designing AI-CDSS interventions for rural clinical contexts in developing countries.
References in corpus (7)
- Unremarkable AI: Fitting Intelligent Decision Support into Critical, Clinical Decision-Making Processes
- Trust in AutoML: Exploring Information Needs for Establishing Trust in Automated Machine Learning Systems
- Calendar.help: Designing a Workflow-Based Scheduling Agent with Humans in the Loop
- AutoDS: Towards Human-Centered Automation of Data Science
- AutoAIViz: Opening the Blackbox of Automated Artificial Intelligence with Conditional Parallel Coordinates
- CASS: Towards Building a Social-Support Chatbot for Online Health Community
- Model LineUpper: Supporting Interactive Model Comparison at Multiple Levels for AutoML