Few-shot Learning for Named Entity Recognition in Medical Text

Abstract

Deep neural network models have recently achieved state-of-the-art performance gains in a variety of natural language processing (NLP) tasks (Young, Hazarika, Poria, & Cambria, 2017). However, these gains rely on the availability of large amounts of annotated examples, without which state-of-the-art performance is rarely achievable. This is especially inconvenient for the many NLP fields where annotated examples are scarce, such as medical text. To improve NLP models in this situation, we evaluate five improvements on named entity recognition (NER) tasks when only ten annotated examples are available: (1) layer-wise initialization with pre-trained weights, (2) hyperparameter tuning, (3) combining pre-training data, (4) custom word embeddings, and (5) optimizing out-of-vocabulary (OOV) words. Experimental results show that the F1 score of 69.3% achievable by state-of-the-art models can be improved to 78.87%.

10 pages, 4 figures, 4 tables

References in corpus (5)

• Enriching Word Vectors with Subword Information
• Multi-source Transfer Learning with Convolutional Neural Networks for Lung Pattern Analysis
• Optimal Hyperparameters for Deep LSTM-Networks for Sequence Labeling Tasks
• Transfer Learning for Video Recognition with Scarce Training Data for Deep Convolutional Neural Network
• Question Answering through Transfer Learning from Large Fine-grained Supervision Data

Cited by in corpus (9)

• Recent Advances in Named Entity Recognition: A Comprehensive Survey and Comparative Study
• From Zero to Hero: Harnessing Transformers for Biomedical Named Entity Recognition in Zero- and Few-shot Contexts
• CLIN-X: pre-trained language models and a study on cross-task transfer for concept extraction in the clinical domain
• Simple and Effective Few-Shot Named Entity Recognition with Structured Nearest Neighbor Learning
• Few-NERD: A Few-Shot Named Entity Recognition Dataset
• Low-Resource Name Tagging Learned with Weakly Labeled Data
• What Can Knowledge Bring to Machine Learning? -- A Survey of Low-shot Learning for Structured Data
• GPT-3 Models are Poor Few-Shot Learners in the Biomedical Domain
• Learning from Language Description: Low-shot Named Entity Recognition via Decomposed Framework