44 citations · 107 across the 9 of their papers we have counts for
11 papers · 1 filter
Where to Pay Attention in Sparse Training for Feature Selection?
Ghada Sokar, Zahra Atashgahi, Mykola Pechenizkiy +1
A new line of research for feature selection based on neural networks has recently emerged. Despite its superiority to classical methods, it requires many training iterations to co…
The Unreasonable Effectiveness of Random Pruning: Return of the Most Naive Baseline for Sparse Training
Shiwei Liu, Tianlong Chen, Xiaohan Chen +4
Random pruning is arguably the most naive way to attain sparsity in neural networks, but has been deemed uncompetitive by either post-training pruning or sparse training. In this p…
Do We Actually Need Dense Over-Parameterization? In-Time Over-Parameterization in Sparse Training
Shiwei Liu, Lu Yin, Decebal Constantin Mocanu +1
In this paper, we introduce a new perspective on training deep neural networks capable of state-of-the-art performance without the need for the expensive over-parameterization by p…
Self-Attention Meta-Learner for Continual Learning
Ghada Sokar, Decebal Constantin Mocanu, Mykola Pechenizkiy
Continual learning aims to provide intelligent agents capable of learning multiple tasks sequentially with neural networks. One of its main challenging, catastrophic forgetting, is…
Learning Invariant Representation for Continual Learning
Ghada Sokar, Decebal Constantin Mocanu, Mykola Pechenizkiy
Continual learning aims to provide intelligent agents that are capable of learning continually a sequence of tasks, building on previously learned knowledge. A key challenge in thi…
Selfish Sparse RNN Training
Shiwei Liu, Decebal Constantin Mocanu, Yulong Pei +1
Sparse neural networks have been widely applied to reduce the computational demands of training and deploying over-parameterized deep neural networks. For inference acceleration, m…