An Information-Theoretic Justification for Model Pruning

Berivan Isik, Tsachy Weissman, Albert No

We study the neural network (NN) compression problem, viewing the tension between the compression ratio and NN performance through the lens of rate-distortion theory. We choose a d…