How much do language models memorize?

John X. Morris, Chawin Sitawarin, Chuan Guo +5

We propose a new method for estimating how much a model knows about a datapoint and use it to measure the capacity of modern language models. Prior studies of language model memori…