Prefix-Tuning for Arbitrary Output Sequences on Pretrained Transformers

Peter Cho-Ho Lam, Ziyi Wang, Zirui Zhou

Prefix-tuning is a parameter-efficient fine-tuning method for large language models (LLMs) that steers model outputs by prepending task-specific prefix representations to the input…