What do we need to build explainable AI systems for the medical domain?

Abstract

Artificial intelligence (AI) generally and machine learning (ML) specifically demonstrate impressive practical success in many different application domains, e.g. in autonomous driving, speech recognition, or recommender systems. Deep learning approaches, trained on extremely large data sets or using reinforcement learning methods have even exceeded human performance in visual tasks, particularly on playing games such as Atari, or mastering the game of Go. Even in the medical domain there are remarkable results. The central problem of such models is that they are regarded as black-box models and even if we understand the underlying mathematical principles, they lack an explicit declarative knowledge representation, hence have difficulty in generating the underlying explanatory structures. This calls for systems enabling to make decisions transparent, understandable and explainable. A huge motivation for our approach are rising legal and privacy aspects. The new European General Data Protection Regulation entering into force on May 25th 2018, will make black-box approaches difficult to use in business. This does not imply a ban on automatic learning approaches or an obligation to explain everything all the time, however, there must be a possibility to make the results re-traceable on demand. In this paper we outline some of our research topics in the context of the relatively new area of explainable-AI with a focus on the application in medicine, which is a very special domain. This is due to the fact that medical professionals are working mostly with distributed heterogeneous and complex sources of data. In this paper we concentrate on three sources: images, *omics data and text. We argue that research in explainable-AI would generally help to facilitate the implementation of AI/ML in the medical domain, and specifically help to facilitate transparency and trust.

This is a survey article and section 3.1. draws heavily from arXiv:1706.07979

References in corpus (3)

• What Does Explainable AI Really Mean? A New Conceptualization of Perspectives
• Towards the Augmented Pathologist: Challenges of Explainable-AI in Digital Pathology
• Deep Learning for Ontology Reasoning

Cited by in corpus (36)

• A Review on Explainable Artificial Intelligence for Healthcare: Why, How, and When?
• Exploring Chemical Space using Natural Language Processing Methodologies for Drug Discovery
• Chatbots as conversational healthcare services
• Post-hoc explanation of black-box classifiers using confident itemsets
• Charting the Sociotechnical Gap in Explainable AI: A Framework to Address the Gap in XAI
• Responsible and Regulatory Conform Machine Learning for Medicine: A Survey of Challenges and Solutions
• Tree-based local explanations of machine learning model predictions, AraucanaXAI
• Directive Explanations for Monitoring the Risk of Diabetes Onset: Introducing Directive Data-Centric Explanations and Combinations to Support What-If Explorations
• Applications of Generative AI in Healthcare: algorithmic, ethical, legal and societal considerations
• Explainable Machine Learning for Predicting Homicide Clearance in the United States
• Explainability Pitfalls: Beyond Dark Patterns in Explainable AI
• Towards Counterfactual and Contrastive Explainability and Transparency of DCNN Image Classifiers
• An Urban Population Health Observatory for Disease Causal Pathway Analysis and Decision Support: Underlying Explainable Artificial Intelligence Model
• Diagnosis of Paratuberculosis in Histopathological Images Based on Explainable Artificial Intelligence and Deep Learning
• Achievements and Challenges in Explaining Deep Learning based Computer-Aided Diagnosis Systems
• Trustworthy Visual Analytics in Clinical Gait Analysis: A Case Study for Patients with Cerebral Palsy
• Stuttering Speech Disfluency Prediction using Explainable Attribution Vectors of Facial Muscle Movements
• SurvLIME-Inf: A simplified modification of SurvLIME for explanation of machine learning survival models
• Explainable Multi-class Classification of the CAMH COVID-19 Mental Health Data
• Interpretable Models Capable of Handling Systematic Missingness in Imbalanced Classes and Heterogeneous Datasets
• Explainability in AI Based Applications: A Framework for Comparing Different Techniques
• On Explainability in AI-Solutions: A Cross-Domain Survey
• On Locality of Local Explanation Models
• Robust Semantic Interpretability: Revisiting Concept Activation Vectors
• On Two XAI Cultures: A Case Study of Non-technical Explanations in Deployed AI System
• Combining Similarity and Adversarial Learning to Generate Visual Explanation: Application to Medical Image Classification
• Machine-Generated Hierarchical Structure of Human Activities to Reveal How Machines Think
• Deep Attention-based Representation Learning for Heart Sound Classification
• Explainable Multi-class Classification of Medical Data
• Achieving Transparency Report Privacy in Linear Time
• Requirement analysis for an artificial intelligence model for the diagnosis of the COVID-19 from chest X-ray data
• A Robust Interpretable Deep Learning Classifier for Heart Anomaly Detection Without Segmentation
• Interpretable Machine Learning Classifiers for Brain Tumour Survival Prediction
• Adaptive Explainable Continual Learning Framework for Regression Problems with Focus on Power Forecasts
• Put Chatbot into Its Interlocutor's Shoes: New Framework to Learn Chatbot Responding with Intention
• Explaining Predictions of Deep Neural Classifier via Activation Analysis