44 citations · 107 across the 18 of their papers we have counts for
5 papers · 1 filter
Sparse Training Theory for Scalable and Efficient Agents
Decebal Constantin Mocanu, Elena Mocanu, Tiago Pinto +5
A fundamental task for artificial intelligence is learning. Deep Neural Networks have proven to cope perfectly with all learning paradigms, i.e. supervised, unsupervised, and reinf…
Do We Actually Need Dense Over-Parameterization? In-Time Over-Parameterization in Sparse Training
Shiwei Liu, Lu Yin, Decebal Constantin Mocanu +1
In this paper, we introduce a new perspective on training deep neural networks capable of state-of-the-art performance without the need for the expensive over-parameterization by p…
Self-Attention Meta-Learner for Continual Learning
Ghada Sokar, Decebal Constantin Mocanu, Mykola Pechenizkiy
Continual learning aims to provide intelligent agents capable of learning multiple tasks sequentially with neural networks. One of its main challenging, catastrophic forgetting, is…
Learning Invariant Representation for Continual Learning
Ghada Sokar, Decebal Constantin Mocanu, Mykola Pechenizkiy
Continual learning aims to provide intelligent agents that are capable of learning continually a sequence of tasks, building on previously learned knowledge. A key challenge in thi…
Selfish Sparse RNN Training
Shiwei Liu, Decebal Constantin Mocanu, Yulong Pei +1
Sparse neural networks have been widely applied to reduce the computational demands of training and deploying over-parameterized deep neural networks. For inference acceleration, m…