KnowPrompt: Knowledge-aware Prompt-tuning with Synergistic Optimization for Relation Extraction

Abstract

Recently, prompt-tuning has achieved promising results for specific few-shot classification tasks. The core idea of prompt-tuning is to insert text pieces (i.e., templates) into the input and transform a classification task into a masked language modeling problem. However, for relation extraction, determining an appropriate prompt template requires domain expertise, and it is cumbersome and time-consuming to obtain a suitable label word. Furthermore, there exists abundant semantic and prior knowledge among the relation labels that cannot be ignored. To this end, we focus on incorporating knowledge among relation labels into prompt-tuning for relation extraction and propose a Knowledge-aware Prompt-tuning approach with synergistic optimization (KnowPrompt). Specifically, we inject latent knowledge contained in relation labels into prompt construction with learnable virtual type words and answer words. Then, we synergistically optimize their representation with structured constraints. Extensive experimental results on five datasets with standard and low-resource settings demonstrate the effectiveness of our approach. Our code and datasets are available in https://github.com/zjunlp/KnowPrompt for reproducibility.

Accepted by WWW2022

References in corpus (10)

• SemEval-2010 Task 8: Multi-Way Classification of Semantic Relations Between Pairs of Nominals
• Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stopping
• GDPNet: Refining Latent Multi-View Graph for Relation Extraction
• Calibrate Before Use: Improving Few-Shot Performance of Language Models
• An Improved Baseline for Sentence-level Relation Extraction
• Prototypical Representation Learning for Relation Extraction
• Improving Relation Extraction with Knowledge-attention
• PTR: Prompt Tuning with Rules for Text Classification
• PRGC: Potential Relation and Global Correspondence Based Joint Relational Triple Extraction
• Dialogue-Based Relation Extraction

Cited by in corpus (31)

• P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks
• Hybrid Transformer with Multi-level Fusion for Multimodal Knowledge Graph Completion
• Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey
• Information Retrieval: Recent Advances and Beyond
• Construction of Knowledge Graphs: State and Challenges
• Differentiable Prompt Makes Pre-trained Language Models Better Few-shot Learners
• Ontology-enhanced Prompt-tuning for Few-shot Learning
• Augmenting Low-Resource Text Classification with Graph-Grounded Pre-training and Prompting
• SentiPrompt: Sentiment Knowledge Enhanced Prompt-Tuning for Aspect-Based Sentiment Analysis
• Few-Shot Bot: Prompt-Based Learning for Dialogue Systems
• Relation Extraction as Open-book Examination: Retrieval-enhanced Prompt Tuning
• PTR: Prompt Tuning with Rules for Text Classification
• Large Knowledge Model: Perspectives and Challenges
• Prompt-and-Align: Prompt-Based Social Alignment for Few-Shot Fake News Detection
• Schema-aware Reference as Prompt Improves Data-Efficient Knowledge Graph Construction
• SoVAR: Building Generalizable Scenarios from Accident Reports for Autonomous Driving Testing
• LLM-Mediated Domain-Specific Voice Agents: The Case of TextileBot
• LiResolver: License Incompatibility Resolution for Open Source Software
• PromptORE -- A Novel Approach Towards Fully Unsupervised Relation Extraction
• AliCG: Fine-grained and Evolvable Conceptual Graph Construction for Semantic Search at Alibaba
• A Knowledge-enhanced Two-stage Generative Framework for Medical Dialogue Information Extraction
• Enhancing Low-Resource Relation Representations through Multi-View Decoupling
• Plug-Tagger: A Pluggable Sequence Labeling Framework Using Language Models
• Maximizing Relation Extraction Potential: A Data-Centric Study to Unveil Challenges and Opportunities
• A Knowledge-Injected Curriculum Pretraining Framework for Question Answering
• Analyzing the Influence of Knowledge Graph Information on Relation Extraction
• Text Mining Drug/Chemical-Protein Interactions using an Ensemble of BERT and T5 Based Models
• EduRiskX: A Neuro-Symbolic Framework with F-Logic Reasoning for Early Academic Risk Prediction
• Simple and Effective Input Reformulations for Translation
• Retrieval-augmented Prompt Learning for Pre-trained Foundation Models
• LOGEN: Few-shot Logical Knowledge-Conditioned Text Generation with Self-training