GRAM: Graph-based Attention Model for Healthcare Representation Learning

Abstract

Deep learning methods exhibit promising performance for predictive modeling in healthcare, but two important challenges remain: -Data insufficiency:Often in healthcare predictive modeling, the sample size is insufficient for deep learning methods to achieve satisfactory results. -Interpretation:The representations learned by deep learning methods should align with medical knowledge. To address these challenges, we propose a GRaph-based Attention Model, GRAM that supplements electronic health records (EHR) with hierarchical information inherent to medical ontologies. Based on the data volume and the ontology structure, GRAM represents a medical concept as a combination of its ancestors in the ontology via an attention mechanism. We compared predictive performance (i.e. accuracy, data needs, interpretability) of GRAM to various methods including the recurrent neural network (RNN) in two sequential diagnoses prediction tasks and one heart failure prediction task. Compared to the basic RNN, GRAM achieved 10% higher accuracy for predicting diseases rarely observed in the training data and 3% improved area under the ROC curve for predicting heart failure using an order of magnitude less training data. Additionally, unlike other methods, the medical concept representations learned by GRAM are well aligned with the medical ontology. Finally, GRAM exhibits intuitive attention behaviors by adaptively generalizing to higher level concepts when facing data insufficiency at the lower level concepts.

References in corpus (12)

• Distributed Representations of Words and Phrases and their Compositionality
• Neural Machine Translation by Jointly Learning to Align and Translate
• Semi-Supervised Classification with Graph Convolutional Networks
• ADADELTA: An Adaptive Learning Rate Method
• Multiple Object Recognition with Visual Attention
• A Simple Way to Initialize Recurrent Networks of Rectified Linear Units
• Learning to Diagnose with LSTM Recurrent Neural Networks
• End-to-end Continuous Speech Recognition using Attention-based Recurrent NN: First Results
• Revisiting Semi-Supervised Learning with Graph Embeddings
• Recurrent Neural Networks for Multivariate Time Series with Missing Values
• Multi-task Prediction of Disease Onsets from Longitudinal Lab Tests
• Deepr: A Convolutional Net for Medical Records

Cited by in corpus (8)

• HealthFog: An Ensemble Deep Learning based Smart Healthcare System for Automatic Diagnosis of Heart Diseases in Integrated IoT and Fog Computing Environments
• RetainVis: Visual Analytics with Interpretable and Interactive Recurrent Neural Networks on Electronic Medical Records
• SECNLP: A Survey of Embeddings in Clinical Natural Language Processing
• ISeeU: Visually interpretable deep learning for mortality prediction inside the ICU
• Graph Kernels: State-of-the-Art and Future Challenges
• Machine Learning for Administrative Health Records: A Systematic Review of Techniques and Applications
• Label Dependent Attention Model for Disease Risk Prediction Using Multimodal Electronic Health Records
• A Literature Review of Recent Graph Embedding Techniques for Biomedical Data