Transfer Learning in Electronic Health Records through Clinical Concept Embedding
arXiv:2107.12919
Abstract
Deep learning models have shown tremendous potential in learning representations, which are able to capture some key properties of the data. This makes them great candidates for transfer learning: Exploiting commonalities between different learning tasks to transfer knowledge from one task to another. Electronic health records (EHR) research is one of the domains that has witnessed a growing number of deep learning techniques employed for learning clinically-meaningful representations of medical concepts (such as diseases and medications). Despite this growth, the approaches to benchmark and assess such learned representations (or, embeddings) is under-investigated; this can be a big issue when such embeddings are shared to facilitate transfer learning. In this study, we aim to (1) train some of the most prominent disease embedding techniques on a comprehensive EHR data from 3.1 million patients, (2) employ qualitative and quantitative evaluation techniques to assess these embeddings, and (3) provide pre-trained disease embeddings for transfer learning. This study can be the first comprehensive approach for clinical concept embedding evaluation and can be applied to any embedding techniques and for any EHR concept.
References in corpus (9)
- Scikit-learn: Machine Learning in Python
- Distributed Representations of Words and Phrases and their Compositionality
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
- Deep EHR: A Survey of Recent Advances in Deep Learning Techniques for Electronic Health Record (EHR) Analysis
- word2vec Explained: deriving Mikolov et al.'s negative-sampling word-embedding method
- Massive Exploration of Neural Machine Translation Architectures
- Med-BERT: pre-trained contextualized embeddings on large-scale structured electronic health records for disease prediction
- Medical Concept Embedding with Time-Aware Attention