35 citations · 70 across the 10 of their papers we have counts for
Showing 2022Show all
2 papers · 1 filter
cs.LG2022
Optimizing Learning Rate Schedules for Iterative Pruning of Deep Neural Networks
Shiyu Liu, Rohan Ghosh, John Tan Chong Min +1
The importance of learning rate (LR) schedules on network pruning has been observed in a few recent works. As an example, Frankle and Carbin (2019) highlighted that winning tickets…
cs.LG2022
AP: Selective Activation for De-sparsifying Pruned Neural Networks
Shiyu Liu, Rohan Ghosh, Dylan Tan +1
The rectified linear unit (ReLU) is a highly successful activation function in neural networks as it allows networks to easily obtain sparse representations, which reduces overfitt…