S-Cyc: A Learning Rate Schedule for Iterative Pruning of ReLU-based Networks

Shiyu Liu, Chong Min John Tan, Mehul Motani

We explore a new perspective on adapting the learning rate (LR) schedule to improve the performance of the ReLU-based network as it is iteratively pruned. Our work and contribution…