What do we need to build explainable AI systems for the medical domain?

Abstract

Artificial intelligence (AI) generally and machine learning (ML) specifically demonstrate impressive practical success in many different application domains, e.g. in autonomous driving, speech recognition, or recommender systems. Deep learning approaches, trained on extremely large data sets or using reinforcement learning methods have even exceeded human performance in visual tasks, particularly on playing games such as Atari, or mastering the game of Go. Even in the medical domain there are remarkable results. The central problem of such models is that they are regarded as black-box models and even if we understand the underlying mathematical principles, they lack an explicit declarative knowledge representation, hence have difficulty in generating the underlying explanatory structures. This calls for systems enabling to make decisions transparent, understandable and explainable. A huge motivation for our approach are rising legal and privacy aspects. The new European General Data Protection Regulation entering into force on May 25th 2018, will make black-box approaches difficult to use in business. This does not imply a ban on automatic learning approaches or an obligation to explain everything all the time, however, there must be a possibility to make the results re-traceable on demand. In this paper we outline some of our research topics in the context of the relatively new area of explainable-AI with a focus on the application in medicine, which is a very special domain. This is due to the fact that medical professionals are working mostly with distributed heterogeneous and complex sources of data. In this paper we concentrate on three sources: images, *omics data and text. We argue that research in explainable-AI would generally help to facilitate the implementation of AI/ML in the medical domain, and specifically help to facilitate transparency and trust.

arXiv commentThis is a survey article and section 3.1. draws heavily from arXiv:1706.07979

References in corpus (5)

• What Does Explainable AI Really Mean? A New Conceptualization of Perspectives
• Generating Visual Explanations
• Towards the Augmented Pathologist: Challenges of Explainable-AI in Digital Pathology
• Deep Learning for Ontology Reasoning
• Cross-modal Deep Metric Learning with Multi-task Regularization

Cited by in corpus (62)

• Deep neural network models for computational histopathology: A survey
• Fast Parallel Hypertree Decompositions in Logarithmic Recursion Depth
• A Review on Explainable Artificial Intelligence for Healthcare: Why, How, and When?
• Exploring Chemical Space using Natural Language Processing Methodologies for Drug Discovery
• Chatbots as conversational healthcare services
• Post-hoc explanation of black-box classifiers using confident itemsets
• GANterfactual - Counterfactual Explanations for Medical Non-Experts using Generative Adversarial Learning
• Explainable Artificial Intelligence and Machine Learning: A reality rooted perspective
• Study of Different Deep Learning Approach with Explainable AI for Screening Patients with COVID-19 Symptoms: Using CT Scan and Chest X-ray Image Dataset
• Charting the Sociotechnical Gap in Explainable AI: A Framework to Address the Gap in XAI
• Interpretable deep learning for guided structure-property explorations in photovoltaics
• On the Explanation of Machine Learning Predictions in Clinical Gait Analysis
• Responsible and Regulatory Conform Machine Learning for Medicine: A Survey of Challenges and Solutions
• Explainable AI, but explainable to whom?
• Ensembles of Convolutional Neural Networks models for pediatric pneumonia diagnosis
• Artificial Intelligence in Glioma Imaging: Challenges and Advances
• Tree-based local explanations of machine learning model predictions, AraucanaXAI
• Checklist for responsible deep learning modeling of medical images based on COVID-19 detection studies
• Directive Explanations for Monitoring the Risk of Diabetes Onset: Introducing Directive Data-Centric Explanations and Combinations to Support What-If Explorations
• A One-Class Classification Decision Tree Based on Kernel Density Estimation
• Applications of Generative AI in Healthcare: algorithmic, ethical, legal and societal considerations
• Kandinsky Patterns
• Explainable Machine Learning for Predicting Homicide Clearance in the United States
• Explainability Pitfalls: Beyond Dark Patterns in Explainable AI
• Counterfactuals and Causability in Explainable Artificial Intelligence: Theory, Algorithms, and Applications
• Foundations of Explainable Knowledge-Enabled Systems
• Towards Counterfactual and Contrastive Explainability and Transparency of DCNN Image Classifiers
• An Urban Population Health Observatory for Disease Causal Pathway Analysis and Decision Support: Underlying Explainable Artificial Intelligence Model
• Saliency is a Possible Red Herring When Diagnosing Poor Generalization
• Health improvement framework for planning actionable treatment process using surrogate Bayesian model
• Diagnosis of Paratuberculosis in Histopathological Images Based on Explainable Artificial Intelligence and Deep Learning
• Enabling Personalized Decision Support with Patient-Generated Data and Attributable Components
• Revisiting the Evaluation of Uncertainty Estimation and Its Application to Explore Model Complexity-Uncertainty Trade-Off
• Achievements and Challenges in Explaining Deep Learning based Computer-Aided Diagnosis Systems
• Neural Temporal Point Processes For Modelling Electronic Health Records
• Trustworthy Visual Analytics in Clinical Gait Analysis: A Case Study for Patients with Cerebral Palsy
• Abstracting Deep Neural Networks into Concept Graphs for Concept Level Interpretability
• Stuttering Speech Disfluency Prediction using Explainable Attribution Vectors of Facial Muscle Movements
• Explainable Multi-class Classification of the CAMH COVID-19 Mental Health Data
• Explainability in AI Based Applications: A Framework for Comparing Different Techniques
• SurvLIME-Inf: A simplified modification of SurvLIME for explanation of machine learning survival models
• Interpretable Models Capable of Handling Systematic Missingness in Imbalanced Classes and Heterogeneous Datasets
• Explainable cardiac pathology classification on cine MRI with motion characterization by semi-supervised learning of apparent flow
• On Locality of Local Explanation Models
• Semantically-aware population health risk analyses
• On Explainability in AI-Solutions: A Cross-Domain Survey
• You Can Do Better! If You Elaborate the Reason When Making Prediction
• Combining Similarity and Adversarial Learning to Generate Visual Explanation: Application to Medical Image Classification
• Robust Semantic Interpretability: Revisiting Concept Activation Vectors
• On Two XAI Cultures: A Case Study of Non-technical Explanations in Deployed AI System
• Deep Attention-based Representation Learning for Heart Sound Classification
• Machine-Generated Hierarchical Structure of Human Activities to Reveal How Machines Think
• Demystifying Brain Tumour Segmentation Networks: Interpretability and Uncertainty Analysis
• Explainable Multi-class Classification of Medical Data
• Explaining Predictions of Deep Neural Classifier via Activation Analysis
• Adaptive Explainable Continual Learning Framework for Regression Problems with Focus on Power Forecasts
• Achieving Transparency Report Privacy in Linear Time
• Put Chatbot into Its Interlocutor's Shoes: New Framework to Learn Chatbot Responding with Intention
• A Robust Interpretable Deep Learning Classifier for Heart Anomaly Detection Without Segmentation
• Parallel Medical Imaging for Intelligent Medical Image Analysis: Concepts, Methods, and Applications
• Requirement analysis for an artificial intelligence model for the diagnosis of the COVID-19 from chest X-ray data
• Interpretable Machine Learning Classifiers for Brain Tumour Survival Prediction