Small-to-Large Generalization: Data Influences Models Consistently Across Scale

Alaa Khaddaj, Logan Engstrom, Aleksander Madry

Choice of training data distribution greatly influences model behavior. Yet, in large-scale settings, precisely characterizing how changes in training data affects predictions is o…