Learning New Facts with QLoRA: An Acquisition-Retention Frontier

Estelle Zheng, Sébastien Warichet, Emmanuel Helbert +1

Parameter-efficient fine-tuning is often assumed to preserve pretrained capabilities because it updates only a small number of parameters. We show that this assumption depends stro…