4.7k citations · 10.6k across the 33 of their papers we have counts for
6 papers · 1 filter
What Do Compressed Deep Neural Networks Forget?
Sara Hooker, Aaron Courville, Gregory Clark +2
Deep neural network pruning and quantization techniques have demonstrated it is possible to achieve high levels of compression with surprisingly little degradation to test set accu…
Simple and Effective Noisy Channel Modeling for Neural Machine Translation
Kyra Yee, Nathan Ng, Yann N. Dauphin +1
Previous work on neural noisy channel modeling relied on latent variable models that incrementally process the source and target sentence. This makes decoding decisions based on pa…
On the Pitfalls of Measuring Emergent Communication
Ryan Lowe, Jakob Foerster, Y-Lan Boureau +2
How do we know if communication is emerging in a multi-agent system? The vast majority of recent papers on emergent communication show that adding a communication channel leads to…
Pay Less Attention with Lightweight and Dynamic Convolutions
Felix Wu, Angela Fan, Alexei Baevski +2
Self-attention is a useful mechanism to build generative models for language and images. It determines the importance of context elements by comparing each element to the current t…
Strategies for Structuring Story Generation
Angela Fan, Mike Lewis, Yann Dauphin
Writers generally rely on plans or sketches to write long stories, but most current language models generate word by word from left to right. We explore coarse-to-fine models for c…
Fixup Initialization: Residual Learning Without Normalization
Hongyi Zhang, Yann N. Dauphin, Tengyu Ma
Normalization layers are a staple in state-of-the-art deep neural network architectures. They are widely believed to stabilize training, enable higher learning rate, accelerate con…