3.1k citations · 5.5k across the 54 of their papers we have counts for
8 papers · 1 filter
Enforcing Encoder-Decoder Modularity in Sequence-to-Sequence Models
Siddharth Dalmia, Abdelrahman Mohamed, Mike Lewis +2
Inspired by modular software design principles of independence, interchangeability, and clarity of interface, we introduce a method for enforcing encoder-decoder modularity in seq2…
Generalization through Memorization: Nearest Neighbor Language Models
Urvashi Khandelwal, Omer Levy, Dan Jurafsky +2
We introduce NN-LMs, which extend a pre-trained neural language model (LM) by linearly interpolating it with a -nearest neighbors (NN) model. The nearest neighbors are com…
BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension
Mike Lewis, Yinhan Liu, Naman Goyal +5
We present BART, a denoising autoencoder for pretraining sequence-to-sequence models. BART is trained by (1) corrupting text with an arbitrary noising function, and (2) learning a…
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Yinhan Liu, Myle Ott, Naman Goyal +7
Language model pretraining has led to significant performance gains but careful comparison between different approaches is challenging. Training is computationally expensive, often…
MelNet: A Generative Model for Audio in the Frequency Domain
Sean Vasquez, Mike Lewis
Capturing high-level structure in audio waveforms is challenging because a single second of audio spans tens of thousands of timesteps. While long-range dependencies are difficult…
Hierarchical Decision Making by Generating and Following Natural Language Instructions
Hengyuan Hu, Denis Yarats, Qucheng Gong +2
We explore using latent natural language instructions as an expressive and compositional representation of complex actions for hierarchical decision making. Rather than directly se…
Improving Semantic Parsing for Task Oriented Dialog
Arash Einolghozati, Panupong Pasupat, Sonal Gupta +4
Semantic parsing using hierarchical representations has recently been proposed for task oriented dialog with promising results [Gupta et al 2018]. In this paper, we present three d…
Strategies for Structuring Story Generation
Angela Fan, Mike Lewis, Yann Dauphin
Writers generally rely on plans or sketches to write long stories, but most current language models generate word by word from left to right. We explore coarse-to-fine models for c…