Learning How to Ask: Querying LMs with Mixtures of Soft Prompts

Abstract

Natural-language prompts have recently been used to coax pretrained language models into performing other AI tasks, using a fill-in-the-blank paradigm (Petroni et al., 2019) or a few-shot extrapolation paradigm (Brown et al., 2020). For example, language models retain factual knowledge from their training corpora that can be extracted by asking them to "fill in the blank" in a sentential prompt. However, where does this prompt come from? We explore the idea of learning prompts by gradient descent -- either fine-tuning prompts taken from previous work, or starting from random initialization. Our prompts consist of "soft words," i.e., continuous vectors that are not necessarily word type embeddings from the language model. Furthermore, for each task, we optimize a mixture of prompts, learning which prompts are most effective and how to ensemble them. Across multiple English LMs and tasks, our approach hugely outperforms previous methods, showing that the implicit factual knowledge in language models was previously underestimated. Moreover, this knowledge is cheap to elicit: random initialization is nearly as good as informed initialization.

arXiv commentNAACL-HLT 2021 camera-ready

References in corpus (2)

• Adam: A Method for Stochastic Optimization
• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks

Cited by in corpus (32)

• On the Opportunities and Risks of Foundation Models
• Pre-trained Models for Natural Language Processing: A Survey
• P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks
• Fuzz4All: Universal Fuzzing with Large Language Models
• Large Language Models for Code: Security Hardening and Adversarial Testing
• Dynamic Prefix-Tuning for Generative Template-based Event Extraction
• Paradigm Shift in Natural Language Processing
• CPT: Colorful Prompt Tuning for Pre-trained Vision-Language Models
• Differentiable Prompt Makes Pre-trained Language Models Better Few-shot Learners
• Model Tuning or Prompt Tuning? A Study of Large Language Models for Clinical Concept and Relation Extraction
• Finetuned Language Models Are Zero-Shot Learners
• Augmenting Low-Resource Text Classification with Graph-Grounded Pre-training and Prompting
• On Transferability of Prompt Tuning for Natural Language Processing
• Parameter-Efficient Transfer Learning with Diff Pruning
• Prompt-and-Align: Prompt-Based Social Alignment for Few-Shot Fake News Detection
• KnowledgeVIS: Interpreting Language Models by Comparing Fill-in-the-Blank Prompts
• Framing the News:From Human Perception to Large Language Model Inferences
• The Life Cycle of Knowledge in Big Language Models: A Survey
• Response Generation with Context-Aware Prompt Learning
• LMSanitator: Defending Prompt-Tuning Against Task-Agnostic Backdoors
• A Few-shot Approach to Resume Information Extraction via Prompts
• Matching Tasks to Objectives: Fine-Tuning and Prompt-Tuning Strategies for Encoder-Decoder Pre-trained Language Models
• Low-Resource Multi-Granularity Academic Function Recognition Based on Multiple Prompt Knowledge
• Inferring Offensiveness In Images From Natural Language Supervision
• Unsupervised Improvement of Factual Knowledge in Language Models
• Collocation2Text: Controllable Text Generation from Guide Phrases in Russian
• Relational World Knowledge Representation in Contextual Language Models: A Review
• Textual Entailment for Effective Triple Validation in Object Prediction
• P-Adapters: Robustly Extracting Factual Information from Language Models with Diverse Prompts
• ModuLoRA: Finetuning 2-Bit LLMs on Consumer GPUs by Integrating with Modular Quantizers
• Simple and Effective Input Reformulations for Translation
• Metric-Driven Reinforcement Learning for Object Hallucination Reduction in LVLM-Based Image Captioning