3.7k citations · 14.2k across the 39 of their papers we have counts for
5 papers · 1 filter
Deep Learning Scaling is Predictable, Empirically
Joel Hestness, Sharan Narang, Newsha Ardalani +6
Deep learning (DL) creates impactful advances following a virtuous recipe: model architecture search, creating large training data sets, and scaling computation. It is widely belie…
Block-Sparse Recurrent Neural Networks
Sharan Narang, Eric Undersander, Gregory Diamos
Recurrent Neural Networks (RNNs) are used in state-of-the-art models in domains such as speech recognition, machine translation, and language modelling. Sparsity is a technique to…
Deep Voice 3: Scaling Text-to-Speech with Convolutional Sequence Learning
Wei Ping, Kainan Peng, Andrew Gibiansky +5
We present Deep Voice 3, a fully-convolutional attention-based neural text-to-speech (TTS) system. Deep Voice 3 matches state-of-the-art neural speech synthesis systems in naturaln…
Mixed Precision Training
Paulius Micikevicius, Sharan Narang, Jonah Alben +8
Deep neural networks have enabled progress in a wide variety of applications. Growing the size of the neural network typically results in improved accuracy. As model sizes grow, th…
Exploring Sparsity in Recurrent Neural Networks
Sharan Narang, Erich Elsen, Gregory Diamos +1
Recurrent Neural Networks (RNN) are widely used to solve a variety of problems and as the quantity of data and the amount of available compute have increased, so have model sizes.…