Disentangling generalization and memorization in large language models using chess

Leonard S. Pleiss, Maximilian Schiffer, Robert K. von Weizsaecker

Large Language Models (LLMs) exhibit remarkable capabilities, yet it remains unclear to what extent these reflect sophisticated recall or genuine reasoning ability. We introduce ch…