1.3k citations · 3.7k across the 43 of their papers we have counts for
13 papers · 1 filter
Learning to Retrieve Passages without Supervision
Ori Ram, Gal Shachaf, Omer Levy +2
Dense retrievers for open-domain question answering (ODQA) have been shown to achieve impressive performance by training on large datasets of question-passage pairs. In this work w…
Simple Local Attentions Remain Competitive for Long-Context Tasks
Wenhan Xiong, Barlas Oğuz, Anchit Gupta +5
Many NLP tasks require processing long contexts beyond the length limit of pretrained models. In order to scale these models to longer text sequences, many efficient long-range att…
A Few More Examples May Be Worth Billions of Parameters
Yuval Kirstain, Patrick Lewis, Sebastian Riedel +1
We investigate the dynamics of increasing the number of model parameters versus the number of labeled examples across a wide variety of tasks. Our exploration reveals that while sc…
ParaShoot: A Hebrew Question Answering Dataset
Omri Keren, Omer Levy
NLP research in Hebrew has largely focused on morphology and syntax, where rich annotated datasets in the spirit of Universal Dependencies are available. Semantic datasets, however…
Models In a Spelling Bee: Language Models Implicitly Learn the Character Composition of Tokens
Itay Itzhak, Omer Levy
Standard pretrained language models operate on sequences of subword tokens without direct access to the characters that compose each token's string representation. We probe the emb…
How Optimal is Greedy Decoding for Extractive Question Answering?
Or Castel, Ori Ram, Avia Efrat +1
Fine-tuned language models use greedy decoding to answer reading comprehension questions with relative success. However, this approach does not ensure that the answer is a span in…