Prompt Compression and Contrastive Conditioning for Controllability and Toxicity Reduction in Language Models

David Wingate, Mohammad Shoeybi, Taylor Sorensen

We explore the idea of compressing the prompts used to condition language models, and show that compressed prompts can retain a substantive amount of information about the original…