From the 1 of 14 linked papers with an AI index.
521 citations · 731 across the 9 of their papers we have counts for
11 papers
Unlocking Compositional Generalization in Pre-trained Models Using Intermediate Representations
Jonathan Herzig, Peter Shaw, Ming-Wei Chang +3
Sequence-to-sequence (seq2seq) models are prevalent in semantic parsing, but have been found to struggle at out-of-distribution compositional generalization. While specialized mode…
Neural Data Augmentation via Example Extrapolation
Kenton Lee, Kelvin Guu, Luheng He +2
In many applications of machine learning, certain categories of examples may be underrepresented in the training data, causing systems to underperform on such "few-shot" cases at t…
NeurIPS 2020 EfficientQA Competition: Systems, Analyses and Lessons Learned
Sewon Min, Jordan Boyd-Graber, Chris Alberti +50
We review the EfficientQA competition from NeurIPS 2020. The competition focused on open-domain question answering (QA), where systems take natural language questions as input and…
Learning Abstract Models for Strategic Exploration and Fast Reward Transfer
Evan Zheran Liu, Ramtin Keramati, Sudarshan Seshadri +4
Model-based reinforcement learning (RL) is appealing because (i) it enables planning and thus more strategic exploration, and (ii) by decoupling dynamics from rewards, it enables f…
Pretraining with Contrastive Sentence Objectives Improves Discourse Performance of Language Models
Dan Iter, Kelvin Guu, Larry Lansing +1
Recent models for unsupervised representation learning of text have employed a number of techniques to improve contextual word representations but have put little focus on discours…
REALM: Retrieval-Augmented Language Model Pre-Training
Kelvin Guu, Kenton Lee, Zora Tung +2
The paper introduces REALM, a language model that retrieves relevant documents from a large corpus during pre‑training and inference, enabling it to use external knowledge for task…