Too Big to Think: Capacity, Memorization, and Generalization in Pre-Trained Transformers

Joshua Barron, Devin White

The relationship between memorization and generalization in large language models (LLMs) remains an open area of research, with growing evidence that the two are deeply intertwined…