Neural Architecture Search for Parameter-Efficient Fine-tuning of Large Pre-trained Language Models

Neal Lawton, Anoop Kumar, Govind Thattai +2

Parameter-efficient tuning (PET) methods fit pre-trained language models (PLMs) to downstream tasks by either computing a small compressed update for a subset of model parameters,…