Clinical Concept Extraction with Contextual Word Embedding

Abstract

Automatic extraction of clinical concepts is an essential step for turning the unstructured data within a clinical note into structured and actionable information. In this work, we propose a clinical concept extraction model for automatic annotation of clinical problems, treatments, and tests in clinical notes utilizing domain-specific contextual word embedding. A contextual word embedding model is first trained on a corpus with a mixture of clinical reports and relevant Wikipedia pages in the clinical domain. Next, a bidirectional LSTM-CRF model is trained for clinical concept extraction using the contextual word embedding model. We tested our proposed model on the I2B2 2010 challenge dataset. Our proposed model achieved the best performance among reported baseline models and outperformed the state-of-the-art models by 3.4% in terms of F1-score.

Machine Learning for Health (ML4H) Workshop at NeurIPS 2018 arXiv:1811.07216

References in corpus (6)

• Adam: A Method for Stochastic Optimization
• Character-Aware Neural Language Models
• Bidirectional LSTM-CRF for Clinical Concept Extraction
• Constituency Parsing with a Self-Attentive Encoder
• CliNER 2.0: Accessible and Accurate Clinical Concept Extraction
• Jointly Predicting Predicates and Arguments in Neural Semantic Role Labeling

Cited by in corpus (13)

• BioBERT: a pre-trained biomedical language representation model for biomedical text mining
• Publicly Available Clinical BERT Embeddings
• Enhancing Clinical Concept Extraction with Contextual Embeddings
• SECNLP: A Survey of Embeddings in Clinical Natural Language Processing
• SciFive: a text-to-text transformer model for biomedical literature
• Probing Biomedical Embeddings from Language Models
• Extracting UMLS Concepts from Medical Text Using General and Domain-Specific Deep Learning Models
• BioNerFlair: biomedical named entity recognition using flair embedding and sequence tagger
• Pre-trained Language Model for Biomedical Question Answering
• Improving Clinical Outcome Predictions Using Convolution over Medical Entities with Multimodal Learning
• Calibrating Structured Output Predictors for Natural Language Processing
• Hierarchical Semantic Correspondence Learning for Post-Discharge Patient Mortality Prediction
• Extracting clinical concepts from user queries