95 citations · 302 across the 15 of their papers we have counts for
19 papers · 1 filter
On the Compositional Generalization Gap of In-Context Learning
Arian Hosseini, Ankit Vani, Dzmitry Bahdanau +2
Pretrained large generative language models have shown great performance on many tasks, but exhibit low compositional generalization abilities. Scaling such models has been shown t…
Evaluating Distributional Distortion in Neural Language Modeling
Benjamin LeBrun, Alessandro Sordoni, Timothy J. O'Donnell
A fundamental characteristic of natural language is the high rate at which speakers produce novel expressions. Because of this novelty, a heavy-tail of rare events accounts for a s…
Better Language Model with Hypernym Class Prediction
He Bai, Tong Wang, Alessandro Sordoni +1
Class-based language models (LMs) have been long devised to address context sparsity in -gram LMs. In this study, we revisit this approach in the context of neural LMs. We hypot…
Self-training with Few-shot Rationalization: Teacher Explanations Aid Student in Few-shot NLU
Meghana Moorthy Bhat, Alessandro Sordoni, Subhabrata Mukherjee
While pre-trained language models have obtained state-of-the-art performance for several natural language understanding tasks, they are quite opaque in terms of their decision-maki…
The Emergence of the Shape Bias Results from Communicative Efficiency
Eva Portelance, Michael C. Frank, Dan Jurafsky +2
By the age of two, children tend to assume that new word categories are based on objects' shape, rather than their color or texture; this assumption is called the shape bias. They…
Understanding by Understanding Not: Modeling Negation in Language Models
Arian Hosseini, Siva Reddy, Dzmitry Bahdanau +3
Negation is a core construction in natural language. Despite being very successful on many tasks, state-of-the-art pre-trained language models often handle negation incorrectly. To…