Longitudinal Pooling & Consistency Regularization to Model Disease Progression from MRIs

Abstract

Many neurological diseases are characterized by gradual deterioration of brain structure and function. Large longitudinal MRI datasets have revealed such deterioration, in part, by applying machine and deep learning to predict diagnosis. A popular approach is to apply Convolutional Neural Networks (CNN) to extract informative features from each visit of the longitudinal MRI and then use those features to classify each visit via Recurrent Neural Networks (RNNs). Such modeling neglects the progressive nature of the disease, which may result in clinically implausible classifications across visits. To avoid this issue, we propose to combine features across visits by coupling feature extraction with a novel longitudinal pooling layer and enforce consistency of the classification across visits in line with disease progression. We evaluate the proposed method on the longitudinal structural MRIs from three neuroimaging datasets: Alzheimer's Disease Neuroimaging Initiative (ADNI, N=404), a dataset composed of 274 normal controls and 329 patients with Alcohol Use Disorder (AUD), and 255 youths from the National Consortium on Alcohol and NeuroDevelopment in Adolescence (NCANDA). In all three experiments our method is superior to other widely used approaches for longitudinal classification thus making a unique contribution towards more accurate tracking of the impact of conditions on the brain. The code is available at https://github.com/ouyangjiahong/longitudinal-pooling.

arXiv commentAccepted by Journal of Biomedical and Health Informatics (JBHI)

References in corpus (10)

• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• On the difficulty of training Recurrent Neural Networks
• Efficient Multi-Scale 3D CNN with Fully Connected CRF for Accurate Brain Lesion Segmentation
• Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
• Convolutional Neural Networks for Classification of Alzheimer's Disease: Overview and Reproducible Evaluation
• Learning to Diagnose with LSTM Recurrent Neural Networks
• Alzheimer's Disease Diagnostics by Adaptation of 3D Convolutional Network
• Training recurrent neural networks robust to incomplete data: application to Alzheimer's disease progression modeling
• A multivariate nonlinear mixed effects model for longitudinal image analysis: Application to amyloid imaging
• Imposing Hard Constraints on Deep Networks: Promises and Limitations