33 citations · 38 across the 4 of their papers we have counts for
10 papers · 1 filter
The Curious Case of Absolute Position Embeddings
Koustuv Sinha, Amirhossein Kazemnejad, Siva Reddy +3
Transformer language models encode the notion of word order using positional information. Most commonly, this positional information is represented by absolute position embeddings…
Evaluating Gender Bias in Natural Language Inference
Shanya Sharma, Manan Dey, Koustuv Sinha
Gender-bias stereotypes have recently raised significant ethical concerns in natural language processing. However, progress in detection and evaluation of gender bias in natural la…
Sometimes We Want Translationese
Prasanna Parthasarathi, Koustuv Sinha, Joelle Pineau +1
Rapid progress in Neural Machine Translation (NMT) systems over the last few years has been driven primarily towards improving translation quality, and as a secondary focus, improv…
Masked Language Modeling and the Distributional Hypothesis: Order Word Matters Pre-training for Little
Koustuv Sinha, Robin Jia, Dieuwke Hupkes +3
A possible explanation for the impressive performance of masked language model (MLM) pre-training is that such models have learned to represent the syntactic structures prevalent i…
UnNatural Language Inference
Koustuv Sinha, Prasanna Parthasarathi, Joelle Pineau +1
Recent investigations into the inner-workings of state-of-the-art large-scale pre-trained Transformer-based Natural Language Understanding (NLU) models indicate that they appear to…
Probing Linguistic Systematicity
Emily Goodwin, Koustuv Sinha, Timothy J. O'Donnell
Recently, there has been much interest in the question of whether deep natural language understanding models exhibit systematicity; generalizing such that units like words make con…