Span-based Joint Entity and Relation Extraction with Transformer Pre-training

Abstract

We introduce SpERT, an attention model for span-based joint entity and relation extraction. Our key contribution is a light-weight reasoning on BERT embeddings, which features entity recognition and filtering, as well as relation classification with a localized, marker-free context representation. The model is trained using strong within-sentence negative samples, which are efficiently extracted in a single BERT pass. These aspects facilitate a search over all spans in the sentence. In ablation studies, we demonstrate the benefits of pre-training, strong negative sampling and localized context. Our model outperforms prior work by up to 2.6% F1 score on several datasets for joint entity and relation extraction.

Published at ECAI 2020; marginally revised version; because of new insights into evaluation metrics used in related work, we updated Table 1 and report both micro/macro averaged entity values for the ADE dataset

References in corpus (6)

• MASS: Masked Sequence to Sequence Pre-training for Language Generation
• A Survey on Recent Advances in Named Entity Recognition from Deep Learning models
• End-to-End Open-Domain Question Answering with BERTserini
• End-to-end neural relation extraction using deep biaffine attention
• Neural Metric Learning for Fast End-to-End Relation Extraction
• A General Framework for Information Extraction using Dynamic Span Graphs

Cited by in corpus (16)

• Structured Prediction as Translation between Augmented Natural Languages
• Imposing Relation Structure in Language-Model Embeddings Using Contrastive Learning
• Two are Better than One: Joint Entity and Relation Extraction with Table-Sequence Encoders
• Jointly Extracting Interventions, Outcomes, and Findings from RCT Reports with LLMs
• Entity and Evidence Guided Relation Extraction for DocRED
• An Information Extraction Study: Take In Mind the Tokenization!
• AutoRC: Improving BERT Based Relation Classification Models via Architecture Search
• Extracting Qualitative Causal Structure with Transformer-Based NLP
• Deeper Task-Specificity Improves Joint Entity and Relation Extraction
• Boosting Span-based Joint Entity and Relation Extraction via Squence Tagging Mechanism
• HySPA: Hybrid Span Generation for Scalable Text-to-Graph Extraction
• A Trigger-Sense Memory Flow Framework for Joint Entity and Relation Extraction
• Joint Extraction of Entity and Relation with Information Redundancy Elimination
• A Data Bootstrapping Recipe for Low Resource Multilingual Relation Classification
• A Conditional Cascade Model for Relational Triple Extraction
• Graph-based Joint Pandemic Concern and Relation Extraction on Twitter