Prototypical Representation Learning for Relation Extraction

Abstract

Recognizing relations between entities is a pivotal task of relational learning. Learning relation representations from distantly-labeled datasets is difficult because of the abundant label noise and complicated expressions in human language. This paper aims to learn predictive, interpretable, and robust relation representations from distantly-labeled data that are effective in different settings, including supervised, distantly supervised, and few-shot learning. Instead of solely relying on the supervision from noisy labels, we propose to learn prototypes for each relation from contextual information to best explore the intrinsic semantics of relations. Prototypes are representations in the feature space abstracting the essential semantics of relations between entities in sentences. We learn prototypes based on objectives with clear geometric interpretation, where the prototypes are unit vectors uniformly dispersed in a unit ball, and statement embeddings are centered at the end of their corresponding prototype vectors on the surface of the ball. This approach allows us to learn meaningful, interpretable prototypes for the final classification. Results on several relation learning tasks show that our model significantly outperforms the previous state-of-the-art models. We further demonstrate the robustness of the encoder and the interpretability of prototypes with extensive experiments.

Accepted by ICLR 2021

References in corpus (9)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Prototypical Networks for Few-shot Learning
• RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space
• Relation Classification via Recurrent Neural Network
• Training Tips for the Transformer Model
• Hypernetwork Knowledge Graph Embeddings
• Reinforcement Learning for Relation Classification from Noisy Data
• Position-aware Self-attention with Relative Positional Encodings for Slot Filling
• RelNN: A Deep Neural Model for Relational Learning

Cited by in corpus (6)

• KnowPrompt: Knowledge-aware Prompt-tuning with Synergistic Optimization for Relation Extraction
• PTR: Prompt Tuning with Rules for Text Classification
• ContrastNER: Contrastive-based Prompt Tuning for Few-shot NER
• MCML: A Novel Memory-based Contrastive Meta-Learning Method for Few Shot Slot Tagging
• Few-NERD: A Few-Shot Named Entity Recognition Dataset
• CLINE: Contrastive Learning with Semantic Negative Examples for Natural Language Understanding