Hierarchically-Refined Label Attention Network for Sequence Labeling

Abstract

CRF has been used as a powerful model for statistical sequence labeling. For neural sequence labeling, however, BiLSTM-CRF does not always lead to better results compared with BiLSTM-softmax local classification. This can be because the simple Markov label transition model of CRF does not give much information gain over strong neural encoding. For better representing label sequences, we investigate a hierarchically-refined label attention network, which explicitly leverages label embeddings and captures potential long-term label dependency by giving each word incrementally refined label distributions with hierarchical attention. Results on POS tagging, NER and CCG supertagging show that the proposed model not only improves the overall tagging accuracy with similar number of parameters, but also significantly speeds up the training and testing compared to BiLSTM-CRF.

EMNLP 2019

References in corpus (6)

• Zero-Shot Learning Through Cross-Modal Transfer
• Optimal Hyperparameters for Deep LSTM-Networks for Sequence Labeling Tasks
• Design Challenges and Misconceptions in Neural Sequence Labeling
• Empower Sequence Labeling with Task-Aware Neural Language Model
• Learning Approximate Inference Networks for Structured Prediction
• Two Local Models for Neural Constituent Parsing

Cited by in corpus (8)

• A Survey on Deep Learning for Named Entity Recognition
• The Temporal Structure of Language Processing in the Human Brain Corresponds to The Layered Hierarchy of Deep Language Models
• A Survey on Recent Advances in Sequence Labeling from Deep Learning Models
• Dialogue State Induction Using Neural Latent Variable Models
• A Co-Interactive Transformer for Joint Slot Filling and Intent Detection
• Speaker-change Aware CRF for Dialogue Act Classification
• Making the Best Use of Review Summary for Sentiment Analysis
• HAN: Higher-order Attention Network for Spoken Language Understanding