Generalization or Memorization? Brittleness Testing for Chess-Trained Language Models

Ethan Tang

Recent work has fine-tuned language models on chess data and reported high benchmark scores as evidence that the resulting models can understand the rules of chess, play full chess…