Achievements and Challenges in Explaining Deep Learning based Computer-Aided Diagnosis Systems

Abstract

Remarkable success of modern image-based AI methods and the resulting interest in their applications in critical decision-making processes has led to a surge in efforts to make such intelligent systems transparent and explainable. The need for explainable AI does not stem only from ethical and moral grounds but also from stricter legislation around the world mandating clear and justifiable explanations of any decision taken or assisted by AI. Especially in the medical context where Computer-Aided Diagnosis can have a direct influence on the treatment and well-being of patients, transparency is of utmost importance for safe transition from lab research to real world clinical practice. This paper provides a comprehensive overview of current state-of-the-art in explaining and interpreting Deep Learning based algorithms in applications of medical research and diagnosis of diseases. We discuss early achievements in development of explainable AI for validation of known disease criteria, exploration of new potential biomarkers, as well as methods for the subsequent correction of AI models. Various explanation methods like visual, textual, post-hoc, ante-hoc, local and global have been thoroughly and critically analyzed. Subsequently, we also highlight some of the remaining challenges that stand in the way of practical applications of AI as a clinical decision support tool and provide recommendations for the direction of future research.

17 pages, 2 figures

References in corpus (21)

• Towards A Rigorous Science of Interpretable Machine Learning
• Methods for Interpreting and Understanding Deep Neural Networks
• Striving for Simplicity: The All Convolutional Net
• A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAI
• Multiple Object Recognition with Visual Attention
• What do we need to build explainable AI systems for the medical domain?
• Sanity Checks for Saliency Maps
• Not Just a Black Box: Learning Important Features Through Propagating Activation Differences
• Visualizing Deep Convolutional Neural Networks Using Natural Pre-Images
• A Benchmark for Interpretability Methods in Deep Neural Networks
• Learn To Pay Attention
• Hierarchical interpretations for neural network predictions
• Explainable Artificial Intelligence: a Systematic Review
• On Interpretability of Deep Learning based Skin Lesion Classifiers using Concept Activation Vectors
• Interpretations are useful: penalizing explanations to align neural networks with prior knowledge
• On quantitative aspects of model interpretability
• A CNN toolbox for skin cancer classification
• A Study on Multimodal and Interactive Explanations for Visual Question Answering
• Deep Adaptive Semantic Logic (DASL): Compiling Declarative Knowledge into Deep Neural Networks
• The Skincare project, an interactive deep learning system for differential diagnosis of malignant skin lesions. Technical Report
• Explaining AI-based Decision Support Systems using Concept Localization Maps