Visualizing Convolutional Networks for MRI-based Diagnosis of Alzheimer's Disease

Abstract

Visualizing and interpreting convolutional neural networks (CNNs) is an important task to increase trust in automatic medical decision making systems. In this study, we train a 3D CNN to detect Alzheimer's disease based on structural MRI scans of the brain. Then, we apply four different gradient-based and occlusion-based visualization methods that explain the network's classification decisions by highlighting relevant areas in the input image. We compare the methods qualitatively and quantitatively. We find that all four methods focus on brain regions known to be involved in Alzheimer's disease, such as inferior and middle temporal gyrus. While the occlusion-based methods focus more on specific regions, the gradient-based methods pick up distributed relevance patterns. Additionally, we find that the distribution of relevance varies across patients, with some having a stronger focus on the temporal lobe, whereas for others more cortical areas are relevant. In summary, we show that applying different visualization methods is important to understand the decisions of a CNN, a step that is crucial to increase clinical impact and trust in computer-based decision support systems.

arXiv commentMLCN 2018

References in corpus (7)

• A Survey on Deep Learning in Medical Image Analysis
• Methods for Interpreting and Understanding Deep Neural Networks
• Predicting Alzheimer's disease: a neuroimaging study with 3D convolutional neural networks
• Alzheimer's Disease Diagnostics by Adaptation of 3D Convolutional Network
• Visual Explanations From Deep 3D Convolutional Neural Networks for Alzheimer's Disease Classification
• 3D CNN-based classification using sMRI and MD-DTI images for Alzheimer disease studies
• Alzheimer's Disease Diagnostics by a Deeply Supervised Adaptable 3D Convolutional Network

Cited by in corpus (7)

• Layer-Wise Relevance Propagation for Explaining Deep Neural Network Decisions in MRI-Based Alzheimer's Disease Classification
• Promises and pitfalls of deep neural networks in neuroimaging-based psychiatric research
• Machine Learning Workflow to Explain Black-box Models for Early Alzheimer's Disease Classification Evaluated for Multiple Datasets
• Understanding 3D CNN Behavior for Alzheimer's Disease Diagnosis from Brain PET Scan
• Harnessing spatial homogeneity of neuroimaging data: patch individual filter layers for CNNs
• Interpreting Age Effects of Human Fetal Brain from Spontaneous fMRI using Deep 3D Convolutional Neural Networks
• Learning Shape Features and Abstractions in 3D Convolutional Neural Networks for Detecting Alzheimer's Disease