TRAP: Understanding and Mitigating Privacy Memorization in Language Models

Muhammed Ustaomeroglu, Ziyue Xu, Hanshen Xiao +3

Fine-tuning a language model on sensitive records can leave it able to reproduce them. We ask when this memorization arises and how to prevent it without knowing in advance which s…