243 citations · 550 across the 9 of their papers we have counts for
4 papers · 1 filter
Powerpropagation: A sparsity inducing weight reparameterisation
Jonathan Schwarz, Siddhant M. Jayakumar, Razvan Pascanu +2
The training of sparse neural networks is becoming an increasingly important tool for reducing the computational footprint of models at training and evaluation, as well enabling th…
Adapting Auxiliary Losses Using Gradient Similarity
Yunshu Du, Wojciech M. Czarnecki, Siddhant M. Jayakumar +3
One approach to deal with the statistical inefficiency of neural networks is to rely on auxiliary losses that help to build useful representations. However, it is not always trivia…
Been There, Done That: Meta-Learning with Episodic Recall
Samuel Ritter, Jane X. Wang, Zeb Kurth-Nelson +4
Meta-learning agents excel at rapidly learning new tasks from open-ended task distributions; yet, they forget what they learn about each task as soon as the next begins. When tasks…
Memory-based Parameter Adaptation
Pablo Sprechmann, Siddhant M. Jayakumar, Jack W. Rae +7
Deep neural networks have excelled on a wide range of problems, from vision to language and game playing. Neural networks very gradually incorporate information into weights as the…