NEURO-DRAM: a 3D recurrent visual attention model for interpretable neuroimaging classification

Abstract

Deep learning is attracting significant interest in the neuroimaging community as a means to diagnose psychiatric and neurological disorders from structural magnetic resonance images. However, there is a tendency amongst researchers to adopt architectures optimized for traditional computer vision tasks, rather than design networks customized for neuroimaging data. We address this by introducing NEURO-DRAM, a 3D recurrent visual attention model tailored for neuroimaging classification. The model comprises an agent which, trained by reinforcement learning, learns to navigate through volumetric images, selectively attending to the most informative regions for a given task. When applied to Alzheimer's disease prediction, NEURODRAM achieves state-of-the-art classification accuracy on an out-of-sample dataset, significantly outperforming a baseline convolutional neural network. When further applied to the task of predicting which patients with mild cognitive impairment will be diagnosed with Alzheimer's disease within two years, the model achieves state-of-the-art accuracy with no additional training. Encouragingly, the agent learns, without explicit instruction, a search policy in agreement with standardized radiological hallmarks of Alzheimer's disease, suggesting a route to automated biomarker discovery for more poorly understood disorders.

Improved network figure

References in corpus (6)

• Recurrent Models of Visual Attention
• Predicting Alzheimer's disease: a neuroimaging study with 3D convolutional neural networks
• Do ImageNet Classifiers Generalize to ImageNet?
• Multimodal and Multiscale Deep Neural Networks for the Early Diagnosis of Alzheimer's Disease using structural MR and FDG-PET images
• Alzheimer's Disease Diagnostics by a Deeply Supervised Adaptable 3D Convolutional Network
• Residual and Plain Convolutional Neural Networks for 3D Brain MRI Classification