On the Predictability of Pruning Across Scales

Jonathan S. Rosenfeld, Jonathan Frankle, Michael Carbin +1

We show that the error of iteratively magnitude-pruned networks empirically follows a scaling law with interpretable coefficients that depend on the architecture and task. We funct…