Scaling Training Data with Lossy Image Compression

Katherine L. Mentzer, Andrea Montanari

Empirically-determined scaling laws have been broadly successful in predicting the evolution of large machine learning models with training data and number of parameters. As a cons…