144 citations · 231 across the 10 of their papers we have counts for
14 papers
muNet: Evolving Pretrained Deep Neural Networks into Scalable Auto-tuning Multitask Systems
Andrea Gesmundo, Jeff Dean
Most uses of machine learning today involve training a model from scratch for a particular task, or sometimes starting with a model pretrained on a related task and then fine-tunin…
Scaling Up Models and Data with and
Adam Roberts, Hyung Won Chung, Anselm Levskaya +40
Recent neural network-based language models have benefited greatly from scaling up the size of training datasets and the number of parameters in the models themselves. Scaling can…
Routing Networks with Co-training for Continual Learning
Mark Collier, Efi Kokiopoulou, Andrea Gesmundo +1
The core challenge with continual learning is catastrophic forgetting, the phenomenon that when neural networks are trained on a sequence of tasks they rapidly forget previously le…
Ranking architectures using meta-learning
Alina Dubatovka, Efi Kokiopoulou, Luciano Sbaiz +3
Neural architecture search has recently attracted lots of research efforts as it promises to automate the manual design of neural networks. However, it requires a large amount of c…
Flexible Multi-task Networks by Learning Parameter Allocation
Krzysztof Maziarz, Efi Kokiopoulou, Andrea Gesmundo +3
This paper proposes a novel learning method for multi-task applications. Multi-task neural networks can learn to transfer knowledge across different tasks by using parameter sharin…
Temporal Coding in Spiking Neural Networks with Alpha Synaptic Function: Learning with Backpropagation
Iulia M. Comsa, Krzysztof Potempa, Luca Versari +3
The timing of individual neuronal spikes is essential for biological brains to make fast responses to sensory stimuli. However, conventional artificial neural networks lack the int…