TENER: Adapting Transformer Encoder for Named Entity Recognition

Abstract

The Bidirectional long short-term memory networks (BiLSTM) have been widely used as an encoder in models solving the named entity recognition (NER) task. Recently, the Transformer is broadly adopted in various Natural Language Processing (NLP) tasks owing to its parallelism and advantageous performance. Nevertheless, the performance of the Transformer in NER is not as good as it is in other NLP tasks. In this paper, we propose TENER, a NER architecture adopting adapted Transformer Encoder to model the character-level features and word-level features. By incorporating the direction and relative distance aware attention and the un-scaled attention, we prove the Transformer-like encoder is just as effective for NER as other NLP tasks.

Corrept typos, update performance based on the public available codes

References in corpus (6)

• Natural Language Processing (almost) from Scratch
• End-to-End Relation Extraction using LSTMs on Sequences and Tree Structures
• Empower Sequence Labeling with Task-Aware Neural Language Model
• CAN-NER: Convolutional Attention Network for Chinese Named Entity Recognition
• Robust Lexical Features for Improved Neural Network Named-Entity Recognition
• Sequence Labeling: A Practical Approach

Cited by in corpus (31)

• MetaFormer Baselines for Vision
• Recent Advances in Named Entity Recognition: A Comprehensive Survey and Comparative Study
• De-identifying Australian Hospital Discharge Summaries: An End-to-End Framework using Ensemble of Deep Learning Models
• A Survey on Recent Advances in Sequence Labeling from Deep Learning Models
• Multilingual Name Entity Recognition and Intent Classification Employing Deep Learning Architectures
• FLAT: Chinese NER Using Flat-Lattice Transformer
• Simplified DOM Trees for Transferable Attribute Extraction from the Web
• TextFlint: Unified Multilingual Robustness Evaluation Toolkit for Natural Language Processing
• Multimodal Indoor Localisation in Parkinson's Disease for Detecting Medication Use: Observational Pilot Study in a Free-Living Setting
• A Unified Generative Framework for Various NER Subtasks
• Improving Named Entity Recognition with Attentive Ensemble of Syntactic Information
• End-to-End Dense Video Grounding via Parallel Regression
• MAWSEO: Adversarial Wiki Search Poisoning for Illicit Online Promotion
• PiSLTRc: Position-informed Sign Language Transformer with Content-aware Convolution
• Speech2Slot: An End-to-End Knowledge-based Slot Filling from Speech
• Accelerating BERT Inference for Sequence Labeling via Early-Exit
• Co-Speech Gesture Detection through Multi-Phase Sequence Labeling
• AutoTrans: Automating Transformer Design via Reinforced Architecture Search
• Named Entity Recognition for Social Media Texts with Semantic Augmentation
• Speaker-change Aware CRF for Dialogue Act Classification
• Multi-Scale Local-Temporal Similarity Fusion for Continuous Sign Language Recognition
• BERT for Monolingual and Cross-Lingual Reverse Dictionary
• KARL-Trans-NER: Knowledge Aware Representation Learning for Named Entity Recognition using Transformers
• rx-anon -- A Novel Approach on the De-Identification of Heterogeneous Data based on a Modified Mondrian Algorithm
• Larger-Context Tagging: When and Why Does It Work?
• TE-ESN: Time Encoding Echo State Network for Prediction Based on Irregularly Sampled Time Series Data
• Cascaded Semantic and Positional Self-Attention Network for Document Classification
• DA-Transformer: Distance-aware Transformer
• Method and Dataset Entity Mining in Scientific Literature: A CNN + Bi-LSTM Model with Self-attention
• A Partition Filter Network for Joint Entity and Relation Extraction
• Less is More: Pre-train a Strong Text Encoder for Dense Retrieval Using a Weak Decoder