The Unreasonable Ineffectiveness of Nucleus Sampling on Mitigating Text Memorization

Luka Borec, Philipp Sadler, David Schlangen

This work analyses the text memorization behavior of large language models (LLMs) when subjected to nucleus sampling. Stochastic decoding methods like nucleus sampling are typicall…