Emergent and Predictable Memorization in Large Language Models

Stella Biderman, USVSN Sai Prashanth, Lintang Sutawika +4

Memorization, or the tendency of large language models (LLMs) to output entire sequences from their training data verbatim, is a key concern for safely deploying language models. I…