Localizing Paragraph Memorization in Language Models

Niklas Stoehr, Mitchell Gordon, Chiyuan Zhang +1

Can we localize the weights and mechanisms used by a language model to memorize and recite entire paragraphs of its training data? In this paper, we show that while memorization is…