11 citations · 20 across the 6 of their papers we have counts for
5 papers · 1 filter
The Curious Case of Absolute Position Embeddings
Koustuv Sinha, Amirhossein Kazemnejad, Siva Reddy +3
Transformer language models encode the notion of word order using positional information. Most commonly, this positional information is represented by absolute position embeddings…
Do Encoder Representations of Generative Dialogue Models Encode Sufficient Information about the Task ?
Prasanna Parthasarathi, Joelle Pineau, Sarath Chandar
Predicting the next utterance in dialogue is contingent on encoding of users' input text to generate appropriate and relevant response in data-driven approaches. Although the seman…
A Brief Study on the Effects of Training Generative Dialogue Models with a Semantic loss
Prasanna Parthasarathi, Mohamed Abdelsalam, Joelle Pineau +1
Neural models trained for next utterance generation in dialogue task learn to mimic the n-gram sequences in the training set with training objectives like negative log-likelihood (…
Sometimes We Want Translationese
Prasanna Parthasarathi, Koustuv Sinha, Joelle Pineau +1
Rapid progress in Neural Machine Translation (NMT) systems over the last few years has been driven primarily towards improving translation quality, and as a secondary focus, improv…
Masked Language Modeling and the Distributional Hypothesis: Order Word Matters Pre-training for Little
Koustuv Sinha, Robin Jia, Dieuwke Hupkes +3
A possible explanation for the impressive performance of masked language model (MLM) pre-training is that such models have learned to represent the syntactic structures prevalent i…