439 citations · 1.2k across the 11 of their papers we have counts for
5 papers · 1 filter
Fast Sparse ConvNets
Erich Elsen, Marat Dukhan, Trevor Gale +1
Historically, the pursuit of efficient inference has been one of the driving forces behind research into new deep learning architectures and building blocks. Some recent examples i…
High Fidelity Speech Synthesis with Adversarial Networks
Mikołaj Bińkowski, Jeff Donahue, Sander Dieleman +5
Generative adversarial networks have seen rapid development in recent years and have led to remarkable improvements in generative modelling of images. However, their application in…
Non-Differentiable Supervised Learning with Evolution Strategies and Hybrid Methods
Karel Lenc, Erich Elsen, Tom Schaul +1
In this work we show that Evolution Strategies (ES) are a viable method for learning non-differentiable parameters of large supervised models. ES are black-box optimization algorit…
The Difficulty of Training Sparse Neural Networks
Utku Evci, Fabian Pedregosa, Aidan Gomez +1
We investigate the difficulties of training sparse neural networks and make new observations about optimization dynamics and the energy landscape within the sparse regime. Recent w…
The State of Sparsity in Deep Neural Networks
Trevor Gale, Erich Elsen, Sara Hooker
We rigorously evaluate three state-of-the-art techniques for inducing sparsity in deep neural networks on two large-scale learning tasks: Transformer trained on WMT 2014 English-to…