What Fits (Into Few Tokens) Doesn't Overfit: Compression and Generalization in ML Research Agents

Martin Andres Bertran, Aaron Roth, Zhiwei Steven Wu

Reusing a held-out benchmark adaptively should, in principle, invite overfitting. Yet benchmark-driven machine learning (ML) has produced surprisingly little overfitting in practic…