SECNLP: A Survey of Embeddings in Clinical Natural Language Processing

Abstract

Traditional representations like Bag of words are high dimensional, sparse and ignore the order as well as syntactic and semantic information. Distributed vector representations or embeddings map variable length text to dense fixed length vectors as well as capture the prior knowledge which can transferred to downstream tasks. Even though embedding has become de facto standard for representations in deep learning based NLP tasks in both general and clinical domains, there is no survey paper which presents a detailed review of embeddings in Clinical Natural Language Processing. In this survey paper, we discuss various medical corpora and their characteristics, medical codes and present a brief overview as well as comparison of popular embeddings models. We classify clinical embeddings into nine types and discuss each embedding type in detail. We discuss various evaluation methods followed by possible solutions to various challenges in clinical embeddings. Finally, we conclude with some of the future directions which will advance the research in clinical embeddings.

arXiv commentPublished in Journal of Biomedical Informatics (For updated version, refer 10.1016/j.jbi.2019.103323)

References in corpus (28)

• Efficient Estimation of Word Representations in Vector Space
• Distributed Representations of Words and Phrases and their Compositionality
• Distributed Representations of Sentences and Documents
• Deep EHR: A Survey of Recent Advances in Deep Learning Techniques for Electronic Health Record (EHR) Analysis
• Do Deep Nets Really Need to be Deep?
• Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
• word2vec Parameter Learning Explained
• Medical Text Classification using Convolutional Neural Networks
• Recurrent neural networks with specialized word embeddings for health-domain named-entity recognition
• Medical Concept Representation Learning from Electronic Health Records and its Application on Heart Failure Prediction
• Sequence Learning with RNNs for Medical Concept Normalization in User-Generated Texts
• GRAM: Graph-based Attention Model for Healthcare Representation Learning
• Classifying medical relations in clinical text via convolutional neural networks
• Semantic Structure and Interpretability of Word Embeddings
• Patient representation learning and interpretable evaluation using clinical notes
• Bidirectional LSTM-CRF for Clinical Concept Extraction
• Clinical Concept Extraction with Contextual Word Embedding
• Exploiting Convolutional Neural Network for Risk Prediction with Medical Feature Embedding
• Bidirectional LSTM-CRF for Clinical Concept Extraction
• Exploiting Task-Oriented Resources to Learn Word Embeddings for Clinical Abbreviation Expansion
• Multi-Label Learning from Medical Plain Text with Convolutional Residual Models
• Retrofitting Concept Vector Representations of Medical Concepts to Improve Estimates of Semantic Similarity and Relatedness
• Measures of dependence between random vectors and tests of independence. Literature review
• Convolutional Neural Networks for Medical Diagnosis from Admission Notes
• Effective Representations of Clinical Notes
• AWE-CM Vectors: Augmenting Word Embeddings with a Clinical Metathesaurus
• Classifying medical notes into standard disease codes using Machine Learning
• Improving Medical Short Text Classification with Semantic Expansion Using Word-Cluster Embedding

Cited by in corpus (7)

• Trustworthy AI: Securing Sensitive Data in Large Language Models
• Pre-trained Language Models in Biomedical Domain: A Systematic Survey
• Integration of Domain Knowledge using Medical Knowledge Graph Deep Learning for Cancer Phenotyping
• Unlocking Electronic Health Records: A Hybrid Graph RAG Approach to Safe Clinical AI for Patient QA
• Explainable Health Risk Predictor with Transformer-based Medicare Claim Encoder
• A reproducible experimental survey on biomedical sentence similarity: a string-based method sets the state of the art
• Computing Conceptual Distances between Breast Cancer Screening Guidelines: An Implementation of a Near-Peer Epistemic Model of Medical Disagreement