Augmented Natural Language for Generative Sequence Labeling

Abstract

We propose a generative framework for joint sequence labeling and sentence-level classification. Our model performs multiple sequence labeling tasks at once using a single, shared natural language output space. Unlike prior discriminative methods, our model naturally incorporates label semantics and shares knowledge across tasks. Our framework is general purpose, performing well on few-shot, low-resource, and high-resource tasks. We demonstrate these advantages on popular named entity recognition, slot labeling, and intent classification benchmarks. We set a new state-of-the-art for few-shot slot labeling, improving substantially upon the previous 5-shot ($75.0\%\to 90.9\%$) and 1-shot ($70.4\%\to 81.0\%$) state-of-the-art results. Furthermore, our model generates large improvements ($46.27\%\to 63.83\%$) in low-resource slot labeling over a BERT baseline by incorporating label semantics. We also maintain competitive results on high-resource tasks, performing within two points of the state-of-the-art on all tasks and setting a new state-of-the-art on the SNIPS dataset.

To appear at EMNLP 2020

References in corpus (7)

• Decoupled Weight Decay Regularization
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Prototypical Networks for Few-shot Learning
• Few-shot classification in Named Entity Recognition Task
• Low-Resource Named Entity Recognition with Cross-Lingual, Character-Level Neural Conditional Random Fields
• Multi-Task Learning for Sequence Tagging: An Empirical Study
• Cross-Lingual Transfer Learning for Multilingual Task Oriented Dialog