95 citations · 417 across the 43 of their papers we have counts for
3 papers · 2 filters
On the Compositional Generalization Gap of In-Context Learning
Arian Hosseini, Ankit Vani, Dzmitry Bahdanau +2
Pretrained large generative language models have shown great performance on many tasks, but exhibit low compositional generalization abilities. Scaling such models has been shown t…
Evaluating Distributional Distortion in Neural Language Modeling
Benjamin LeBrun, Alessandro Sordoni, Timothy J. O'Donnell
A fundamental characteristic of natural language is the high rate at which speakers produce novel expressions. Because of this novelty, a heavy-tail of rare events accounts for a s…
Better Language Model with Hypernym Class Prediction
He Bai, Tong Wang, Alessandro Sordoni +1
Class-based language models (LMs) have been long devised to address context sparsity in -gram LMs. In this study, we revisit this approach in the context of neural LMs. We hypot…