2.2k citations · 5.1k across the 21 of their papers we have counts for
6 papers · 1 filter
Deep Learning Scaling is Predictable, Empirically
Joel Hestness, Sharan Narang, Newsha Ardalani +6
Deep learning (DL) creates impactful advances following a virtuous recipe: model architecture search, creating large training data sets, and scaling computation. It is widely belie…
Block-Sparse Recurrent Neural Networks
Sharan Narang, Eric Undersander, Gregory Diamos
Recurrent Neural Networks (RNNs) are used in state-of-the-art models in domains such as speech recognition, machine translation, and language modelling. Sparsity is a technique to…
Mixed Precision Training
Paulius Micikevicius, Sharan Narang, Jonah Alben +8
Deep neural networks have enabled progress in a wide variety of applications. Growing the size of the neural network typically results in improved accuracy. As model sizes grow, th…
Deep Voice 2: Multi-Speaker Neural Text-to-Speech
Sercan Arik, Gregory Diamos, Andrew Gibiansky +5
We introduce a technique for augmenting neural text-to-speech (TTS) with lowdimensional trainable speaker embeddings to generate different voices from a single model. As a starting…
Exploring Sparsity in Recurrent Neural Networks
Sharan Narang, Erich Elsen, Gregory Diamos +1
Recurrent Neural Networks (RNN) are widely used to solve a variety of problems and as the quantity of data and the amount of available compute have increased, so have model sizes.…
Deep Voice: Real-time Neural Text-to-Speech
Sercan O. Arik, Mike Chrzanowski, Adam Coates +9
We present Deep Voice, a production-quality text-to-speech system constructed entirely from deep neural networks. Deep Voice lays the groundwork for truly end-to-end neural speech…