3.1k citations · 3.6k across the 29 of their papers we have counts for
36 papers
Autoregressive Search Engines: Generating Substrings as Document Identifiers
Michele Bevilacqua, Giuseppe Ottaviano, Patrick Lewis +3
Knowledge-intensive language tasks require NLP systems to both provide the correct answer and retrieve supporting evidence for it in a given corpus. Autoregressive language models…
DiffCSE: Difference-based Contrastive Learning for Sentence Embeddings
Yung-Sung Chuang, Rumen Dangovski, Hongyin Luo +7
We propose DiffCSE, an unsupervised contrastive learning framework for learning sentence embeddings. DiffCSE learns sentence embeddings that are sensitive to the difference between…
Salient Phrase Aware Dense Retrieval: Can a Dense Retriever Imitate a Sparse One?
Xilun Chen, Kushal Lakhotia, Barlas Oğuz +6
Despite their recent popularity and well-known advantages, dense retrievers still lag behind sparse methods such as BM25 in their ability to reliably match salient phrases and rare…
Domain-matched Pre-training Tasks for Dense Retrieval
Barlas Oğuz, Kushal Lakhotia, Anchit Gupta +8
Pre-training on larger datasets with ever increasing model size is now a proven recipe for increased performance across almost all NLP tasks. A notable exception is information ret…
NeurIPS 2020 EfficientQA Competition: Systems, Analyses and Lessons Learned
Sewon Min, Jordan Boyd-Graber, Chris Alberti +50
We review the EfficientQA competition from NeurIPS 2020. The competition focused on open-domain question answering (QA), where systems take natural language questions as input and…
Multi-task Retrieval for Knowledge-Intensive Tasks
Jean Maillard, Vladimir Karpukhin, Fabio Petroni +4
Retrieving relevant contexts from a large corpus is a crucial step for tasks such as open-domain question answering and fact checking. Although neural retrieval outperforms traditi…
Studying Strategically: Learning to Mask for Closed-book QA
Qinyuan Ye, Belinda Z. Li, Sinong Wang +5
Closed-book question-answering (QA) is a challenging task that requires a model to directly answer questions without access to external knowledge. It has been shown that directly f…
FiD-Ex: Improving Sequence-to-Sequence Models for Extractive Rationale Generation
Kushal Lakhotia, Bhargavi Paranjape, Asish Ghoshal +3
Natural language (NL) explanations of model predictions are gaining popularity as a means to understand and verify decisions made by large black-box pre-trained models, for NLP tas…
Joint Verification and Reranking for Open Fact Checking Over Tables
Michael Schlichtkrull, Vladimir Karpukhin, Barlas Oğuz +3
Structured information is an important knowledge source for automatic verification of factual claims. Nevertheless, the majority of existing research into this task has focused on…
RECONSIDER: Re-Ranking using Span-Focused Cross-Attention for Open Domain Question Answering
Srinivasan Iyer, Sewon Min, Yashar Mehdad +1
State-of-the-art Machine Reading Comprehension (MRC) models for Open-domain Question Answering (QA) are typically trained for span selection using distantly supervised positive exa…
Efficient One-Pass End-to-End Entity Linking for Questions
Belinda Z. Li, Sewon Min, Srinivasan Iyer +2
We present ELQ, a fast end-to-end entity linking model for questions, which uses a biencoder to jointly perform mention detection and linking in one pass. Evaluated on WebQSP and G…
Answering Complex Open-Domain Questions with Multi-Hop Dense Retrieval
Wenhan Xiong, Xiang Lorraine Li, Srini Iyer +8
We propose a simple and efficient multi-hop dense retrieval approach for answering complex open-domain questions, which achieves state-of-the-art performance on two multi-hop datas…
Language Models as Fact Checkers?
Nayeon Lee, Belinda Z. Li, Sinong Wang +3
Recent work has suggested that language models (LMs) store both common-sense and factual knowledge learned from pre-training data. In this paper, we leverage this implicit knowledg…
TaBERT: Pretraining for Joint Understanding of Textual and Tabular Data
Pengcheng Yin, Graham Neubig, Wen-tau Yih +1
Recent years have witnessed the burgeoning of pretrained language models (LMs) for text-based natural language (NL) understanding tasks. Such models are typically trained on free-f…
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Patrick Lewis, Ethan Perez, Aleksandra Piktus +9
Large pre-trained language models have been shown to store factual knowledge in their parameters, and achieve state-of-the-art results when fine-tuned on downstream NLP tasks. Howe…
An Imitation Game for Learning Semantic Parsers from User Interaction
Ziyu Yao, Yiqi Tang, Wen-tau Yih +2
Despite the widely successful applications, bootstrapping and fine-tuning semantic parsers are still a tedious process with challenges such as costly data annotation and privacy ri…
Dense Passage Retrieval for Open-Domain Question Answering
Vladimir Karpukhin, Barlas Oğuz, Sewon Min +5
Open-domain question answering relies on efficient passage retrieval to select candidate contexts, where traditional sparse vector space models, such as TF-IDF or BM25, are the de…
Unsupervised Question Decomposition for Question Answering
Ethan Perez, Patrick Lewis, Wen-tau Yih +2
We aim to improve question answering (QA) by decomposing hard questions into simpler sub-questions that existing QA systems are capable of answering. Since labeling questions with…
Model-based Interactive Semantic Parsing: A Unified Framework and A Text-to-SQL Case Study
Ziyu Yao, Yu Su, Huan Sun +1
As a promising paradigm, interactive semantic parsing has shown to improve both semantic parsing accuracy and user confidence in the results. In this paper, we propose a new, unifi…
Everything Happens for a Reason: Discovering the Purpose of Actions in Procedural Text
Bhavana Dalvi Mishra, Niket Tandon, Antoine Bosselut +2
Our goal is to better comprehend procedural text, e.g., a paragraph about photosynthesis, by not only predicting what happens, but why some actions need to happen before others. Ou…
Be Consistent! Improving Procedural Text Comprehension using Label Consistency
Xinya Du, Bhavana Dalvi Mishra, Niket Tandon +4
Our goal is procedural text comprehension, namely tracking how the properties of entities (e.g., their location) change with time given a procedural text (e.g., a paragraph about p…
QuaRel: A Dataset and Models for Answering Questions about Qualitative Relationships
Oyvind Tafjord, Peter Clark, Matt Gardner +2
Many natural language questions require recognizing and reasoning with qualitative relationships (e.g., in science, economics, and medicine), but are challenging to answer with cor…
Dissecting Contextual Word Embeddings: Architecture and Representation
Matthew E. Peters, Mark Neumann, Luke Zettlemoyer +1
Contextual word representations derived from pre-trained bidirectional language models (biLMs) have recently been shown to provide significant improvements to the state of the art…
Policy Shaping and Generalized Update Equations for Semantic Parsing from Denotations
Dipendra Misra, Ming-Wei Chang, Xiaodong He +1
Semantic parsing from denotations faces two key challenges in model training: (1) given only the denotations (e.g., answers), search for good candidate semantic parses, and (2) cho…