Full deep neural network training on a pruned weight budget

Maximilian Golub, Guy Lemieux, Mieszko Lis

We introduce a DNN training technique that learns only a fraction of the full parameter set without incurring an accuracy penalty. To do this, our algorithm constrains the total nu…