Mitigating Unintended Memorization in Language Models via Alternating Teaching

Zhe Liu, Xuedong Zhang, Fuchun Peng

Recent research has shown that language models have a tendency to memorize rare or unique sequences in the training corpora which can thus leak sensitive attributes of user data. W…