MT-BioNER: Multi-task Learning for Biomedical Named Entity Recognition using Deep Bidirectional Transformers

Abstract

Conversational agents such as Cortana, Alexa and Siri are continuously working on increasing their capabilities by adding new domains. The support of a new domain includes the design and development of a number of NLU components for domain classification, intents classification and slots tagging (including named entity recognition). Each component only performs well when trained on a large amount of labeled data. Second, these components are deployed on limited-memory devices which requires some model compression. Third, for some domains such as the health domain, it is hard to find a single training data set that covers all the required slot types. To overcome these mentioned problems, we present a multi-task transformer-based neural architecture for slot tagging. We consider the training of a slot tagger using multiple data sets covering different slot types as a multi-task learning problem. The experimental results on the biomedical domain have shown that the proposed approach outperforms the previous state-of-the-art systems for slot tagging on the different benchmark biomedical datasets in terms of (time and memory) efficiency and effectiveness. The output slot tagger can be used by the conversational agent to better identify entities in the input utterances.

References in corpus (6)

• Distributed Representations of Words and Phrases and their Compositionality
• BioBERT: a pre-trained biomedical language representation model for biomedical text mining
• Publicly Available Clinical BERT Embeddings
• ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmission
• Multi-Task Deep Neural Networks for Natural Language Understanding
• Audio De-identification: A New Entity Recognition Task

Cited by in corpus (7)

• Comparison of biomedical relationship extraction methods and models for knowledge graph creation
• AMMUS : A Survey of Transformer-based Pretrained Models in Natural Language Processing
• Pre-trained Language Models in Biomedical Domain: A Systematic Survey
• CLIN-X: pre-trained language models and a study on cross-task transfer for concept extraction in the clinical domain
• Multilingual Name Entity Recognition and Intent Classification Employing Deep Learning Architectures
• An Empirical Study of Multi-Task Learning on BERT for Biomedical Text Mining
• rx-anon -- A Novel Approach on the De-Identification of Heterogeneous Data based on a Modified Mondrian Algorithm