Transfer Learning for Named-Entity Recognition with Neural Networks

Abstract

Recent approaches based on artificial neural networks (ANNs) have shown promising results for named-entity recognition (NER). In order to achieve high performances, ANNs need to be trained on a large labeled dataset. However, labels might be difficult to obtain for the dataset on which the user wants to perform NER: label scarcity is particularly pronounced for patient note de-identification, which is an instance of NER. In this work, we analyze to what extent transfer learning may address this issue. In particular, we demonstrate that transferring an ANN model trained on a large labeled dataset to another dataset with a limited number of labels improves upon the state-of-the-art results on two different datasets for patient note de-identification.

The first two authors contributed equally to this work

References in corpus (3)

• How Transferable are Neural Networks in NLP Applications?
• Transfer Learning for Speech and Language Processing
• Transfer learning approach for financial applications

Cited by in corpus (11)

• A Survey on Deep Learning for Named Entity Recognition
• A Survey on Transfer Learning in Natural Language Processing
• Automatic end-to-end De-identification: Is high accuracy the only metric?
• Cross-domain Named Entity Recognition via Graph Matching
• Recent Advances in Named Entity Recognition: A Comprehensive Survey and Comparative Study
• Transfer Learning for Clinical Time Series Analysis using Recurrent Neural Networks
• CLIN-X: pre-trained language models and a study on cross-task transfer for concept extraction in the clinical domain
• Semi-Supervised Disentangled Framework for Transferable Named Entity Recognition
• Cross-Domain Contract Element Extraction with a Bi-directional Feedback Clause-Element Relation Network
• A survey of automatic de-identification of longitudinal clinical narratives
• Neural Adaptation Layers for Cross-domain Named Entity Recognition