Machine Learning Model Sizes and the Parameter Gap

Pablo Villalobos, Jaime Sevilla, Tamay Besiroglu +3

We study trends in model size of notable machine learning systems over time using a curated dataset. From 1950 to 2018, model size in language models increased steadily by seven or…