BERT-based Ranking for Biomedical Entity Normalization

Abstract

Developing high-performance entity normalization algorithms that can alleviate the term variation problem is of great interest to the biomedical community. Although deep learning-based methods have been successfully applied to biomedical entity normalization, they often depend on traditional context-independent word embeddings. Bidirectional Encoder Representations from Transformers (BERT), BERT for Biomedical Text Mining (BioBERT) and BERT for Clinical Text Mining (ClinicalBERT) were recently introduced to pre-train contextualized word representation models using bidirectional Transformers, advancing the state-of-the-art for many natural language processing tasks. In this study, we proposed an entity normalization architecture by fine-tuning the pre-trained BERT / BioBERT / ClinicalBERT models and conducted extensive experiments to evaluate the effectiveness of the pre-trained models for biomedical entity normalization using three different types of datasets. Our experimental results show that the best fine-tuned models consistently outperformed previous methods and advanced the state-of-the-art for biomedical entity normalization, with up to 1.17% increase in accuracy.

9 pages, 1 figure, 4 tables

References in corpus (2)

• BioBERT: a pre-trained biomedical language representation model for biomedical text mining
• Enhancing Clinical Concept Extraction with Contextual Embeddings

Cited by in corpus (8)

• Transformers in Healthcare: A Survey
• Improving Broad-Coverage Medical Entity Linking with Semantic Type Prediction and Large-Scale Datasets
• Application of Deep Learning in Generating Structured Radiology Reports: A Transformer-Based Technique
• Extracting Concepts for Precision Oncology from the Biomedical Literature
• A Large Language Model Outperforms Other Computational Approaches to the High-Throughput Phenotyping of Physician Notes
• A multi-perspective combined recall and rank framework for Chinese procedure terminology normalization
• Named Entity Normalization Model Using Edge Weight Updating Neural Network: Assimilation Between Knowledge-Driven Graph and Data-Driven Graph
• RadLex Normalization in Radiology Reports