PTR: Prompt Tuning with Rules for Text Classification

Abstract

Fine-tuned pre-trained language models (PLMs) have achieved awesome performance on almost all NLP tasks. By using additional prompts to fine-tune PLMs, we can further stimulate the rich knowledge distributed in PLMs to better serve downstream tasks. Prompt tuning has achieved promising results on some few-class classification tasks such as sentiment classification and natural language inference. However, manually designing lots of language prompts is cumbersome and fallible. For those auto-generated prompts, it is also expensive and time-consuming to verify their effectiveness in non-few-shot scenarios. Hence, it is still challenging for prompt tuning to address many-class classification tasks. To this end, we propose prompt tuning with rules (PTR) for many-class text classification and apply logic rules to construct prompts with several sub-prompts. In this way, PTR is able to encode prior knowledge of each class into prompt tuning. We conduct experiments on relation classification, a typical and complicated many-class classification task, and the results show that PTR can significantly and consistently outperform existing state-of-the-art baselines. This indicates that PTR is a promising approach to take advantage of both human prior knowledge and PLMs for those complicated classification tasks.

References in corpus (9)

• Pre-trained Models for Natural Language Processing: A Survey
• KnowPrompt: Knowledge-aware Prompt-tuning with Synergistic Optimization for Relation Extraction
• A Simple Method for Commonsense Reasoning
• Prefix-Tuning: Optimizing Continuous Prompts for Generation
• Towards a Human-like Open-Domain Chatbot
• Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stopping
• Making Pre-trained Language Models Better Few-shot Learners
• An Improved Baseline for Sentence-level Relation Extraction
• Prototypical Representation Learning for Relation Extraction

Cited by in corpus (12)

• KnowPrompt: Knowledge-aware Prompt-tuning with Synergistic Optimization for Relation Extraction
• P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks
• Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey
• Paradigm Shift in Natural Language Processing
• Differentiable Prompt Makes Pre-trained Language Models Better Few-shot Learners
• SentiPrompt: Sentiment Knowledge Enhanced Prompt-Tuning for Aspect-Based Sentiment Analysis
• LFPT5: A Unified Framework for Lifelong Few-shot Language Learning Based on Prompt Tuning of T5
• NSP-BERT: A Prompt-based Few-Shot Learner Through an Original Pre-training Task--Next Sentence Prediction
• Exploring Prompt-based Few-shot Learning for Grounded Dialog Generation
• LM-CPPF: Paraphrasing-Guided Data Augmentation for Contrastive Prompt-Based Few-Shot Fine-Tuning
• Pre-Trained Models: Past, Present and Future
• Plug-Tagger: A Pluggable Sequence Labeling Framework Using Language Models