Quantifying Memorization Across Neural Language Models

Abstract

Large language models (LMs) have been shown to memorize parts of their training data, and when prompted appropriately, they will emit the memorized training data verbatim. This is undesirable because memorization violates privacy (exposing user data), degrades utility (repeated easy-to-memorize text is often low quality), and hurts fairness (some texts are memorized over others). We describe three log-linear relationships that quantify the degree to which LMs emit memorized training data. Memorization significantly grows as we increase (1) the capacity of a model, (2) the number of times an example has been duplicated, and (3) the number of tokens of context used to prompt the model. Surprisingly, we find the situation becomes more complicated when generalizing these results across model families. On the whole, we find that memorization in LMs is more prevalent than previously believed and will likely get worse as models continues to scale, at least without active mitigations.

Cited by in corpus (16)

• Large Legal Fictions: Profiling Legal Hallucinations in Large Language Models
• Large AI Models in Health Informatics: Applications, Challenges, and the Future
• The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Dataset
• The Ethical Implications of Generative Audio Models: A Systematic Literature Review
• Stronger Together: on the Articulation of Ethical Charters, Legal Tools, and Technical Documentation in ML
• Mapping the individual, social, and biospheric impacts of Foundation Models
• Findings of the The RuATD Shared Task 2022 on Artificial Text Detection in Russian
• What makes a language easy to deep-learn? Deep neural networks and humans similarly benefit from compositional structure
• ByGPT5: End-to-End Style-conditioned Poetry Generation with Token-free Language Models
• LLMs and Stack Overflow Discussions: Reliability, Impact, and Challenges
• SingSong: Generating musical accompaniments from singing
• Normative Conflicts and Shallow AI Alignment
• LLM-ProS: Analyzing Large Language Models' Performance in Competitive Problem Solving
• Fundamental Limits of Membership Inference Attacks on Machine Learning Models
• Pretraining Data Exposure in Large Language Models: A Survey of Membership Inference, Data Contamination, and Security Implications
• Privacy-Aware Document Visual Question Answering