Augmenting Biomedical Named Entity Recognition with General-domain Resources

Abstract

Training a neural network-based biomedical named entity recognition (BioNER) model usually requires extensive and costly human annotations. While several studies have employed multi-task learning with multiple BioNER datasets to reduce human effort, this approach does not consistently yield performance improvements and may introduce label ambiguity in different biomedical corpora. We aim to tackle those challenges through transfer learning from easily accessible resources with fewer concept overlaps with biomedical datasets. We proposed GERBERA, a simple-yet-effective method that utilized general-domain NER datasets for training. We performed multi-task learning to train a pre-trained biomedical language model with both the target BioNER dataset and the general-domain dataset. Subsequently, we fine-tuned the models specifically for the BioNER dataset. We systematically evaluated GERBERA on five datasets of eight entity types, collectively consisting of 81,410 instances. Despite using fewer biomedical resources, our models demonstrated superior performance compared to baseline models trained with additional BioNER datasets. Specifically, our models consistently outperformed the baseline models in six out of eight entity types, achieving an average improvement of 0.9% over the best baseline performance across eight entities. Our method was especially effective in amplifying performance on BioNER datasets characterized by limited data, with a 4.7% improvement in F1 scores on the JNLPBA-RNA dataset. This study introduces a new training method that leverages cost-effective general-domain NER datasets to augment BioNER models. This approach significantly improves BioNER model performance, making it a valuable asset for scenarios with scarce or costly biomedical datasets.

Published in JBI 2024. We make data, codes, and models publicly available via https://github.com/qingyu-qc/bioner_gerbera

References in corpus (10)

• BioBERT: a pre-trained biomedical language representation model for biomedical text mining
• Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing
• BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Mining
• BioRED: A Rich Biomedical Relation Extraction Dataset
• BERT Based Clinical Knowledge Extraction for Biomedical Knowledge Graph Construction and Analysis
• BERN2: an advanced neural biomedical named entity recognition and normalization tool
• CollaboNet: collaboration of deep neural networks for biomedical named entity recognition
• AIONER: All-in-one scheme-based biomedical named entity recognition using deep learning
• How Do Your Biomedical Named Entity Recognition Models Generalize to Novel Entities?
• Improving Tagging Consistency and Entity Coverage for Chemical Identification in Full-text Articles