Probabilistic fine-tuning of pruning masks and PAC-Bayes self-bounded learning

Soufiane Hayou, Bobby He, Gintare Karolina Dziugaite

We study an approach to learning pruning masks by optimizing the expected loss of stochastic pruning masks, i.e., masks which zero out each weight independently with some weight-sp…