Efficient and Private: Memorisation under differentially private parameter-efficient fine-tuning in language models

Olivia Ma, Jonathan Passerat-Palmbach, Dmitrii Usynin

Fine-tuning large language models (LLMs) for specific tasks introduces privacy risks, as models may inadvertently memorise and leak sensitive training data. While Differential Priv…