3.1k citations · 4.1k across the 30 of their papers we have counts for
24 papers · 1 filter
EMBL AI Librarian: Life-Sciences Knowledge Layer for AI Agents
Luigi Sigillo, Matteo Silvestri, Francesco Tabaro +9
The web is increasingly accessed by AI agents rather than humans. Every agent needs knowledge, especially in the life-sciences, where agentic pipelines are growing fast. Access to…
Lost in the Middle: How Language Models Use Long Contexts
Nelson F. Liu, Kevin Lin, John Hewitt +4
While recent language models have the ability to take long contexts as input, relatively little is known about how well they use longer context. We analyze the performance of langu…
Can discrete information extraction prompts generalize across language models?
Nathanaël Carraz Rakotonirina, Roberto Dessì, Fabio Petroni +2
We study whether automatically-induced prompts that effectively extract information from a language model can also be used, out-of-the-box, to probe other language models for the s…
EditEval: An Instruction-Based Benchmark for Text Improvements
Jane Dwivedi-Yu, Timo Schick, Zhengbao Jiang +6
Evaluation of text generation to date has primarily focused on content created sequentially, rather than improvements on a piece of text. Writing, however, is naturally an iterativ…
PEER: A Collaborative Language Model
Timo Schick, Jane Dwivedi-Yu, Zhengbao Jiang +7
Textual content is often the output of a collaborative writing process: We start with an initial draft, ask for suggestions, and repeatedly make changes. Agnostic of this process,…
Atlas: Few-shot Learning with Retrieval Augmented Language Models
Gautier Izacard, Patrick Lewis, Maria Lomeli +7
Large language models have shown impressive few-shot results on a wide range of tasks. However, when knowledge is key for such results, as is the case for tasks such as question an…
EDIN: An End-to-end Benchmark and Pipeline for Unknown Entity Discovery and Indexing
Nora Kassner, Fabio Petroni, Mikhail Plekhanov +2
Existing work on Entity Linking mostly assumes that the reference knowledge base is complete, and therefore all mentions can be linked. In practice this is hardly ever the case, as…
Open Vocabulary Extreme Classification Using Generative Models
Daniel Simig, Fabio Petroni, Pouya Yanki +4
The extreme multi-label classification (XMC) task aims at tagging content with a subset of labels from an extremely large label set. The label vocabulary is typically defined in ad…
Autoregressive Search Engines: Generating Substrings as Document Identifiers
Michele Bevilacqua, Giuseppe Ottaviano, Patrick Lewis +3
Knowledge-intensive language tasks require NLP systems to both provide the correct answer and retrieve supporting evidence for it in a given corpus. Autoregressive language models…
Boosted Dense Retriever
Patrick Lewis, Barlas Oğuz, Wenhan Xiong +3
We propose DrBoost, a dense retrieval ensemble inspired by boosting. DrBoost is trained in stages: each component model is learned sequentially and specialized by focusing only on…
The Web Is Your Oyster - Knowledge-Intensive NLP against a Very Large Web Corpus
Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin +8
In order to address increasing demands of real-world applications, the research for knowledge-intensive NLP (KI-NLP) should advance by capturing the challenges of a truly open-doma…
Cutting Down on Prompts and Parameters: Simple Few-Shot Learning with Language Models
Robert L. Logan, Ivana Balažević, Eric Wallace +3
Prompting language models (LMs) with training examples and task descriptions has been seen as critical to recent successes in few-shot learning. In this work, we show that finetuni…
Multilingual Autoregressive Entity Linking
Nicola De Cao, Ledell Wu, Kashyap Popat +7
We present mGENRE, a sequence-to-sequence system for the Multilingual Entity Linking (MEL) problem -- the task of resolving language-specific mentions to a multilingual Knowledge B…
NeurIPS 2020 EfficientQA Competition: Systems, Analyses and Lessons Learned
Sewon Min, Jordan Boyd-Graber, Chris Alberti +50
We review the EfficientQA competition from NeurIPS 2020. The competition focused on open-domain question answering (QA), where systems take natural language questions as input and…
Multi-task Retrieval for Knowledge-Intensive Tasks
Jean Maillard, Vladimir Karpukhin, Fabio Petroni +4
Retrieving relevant contexts from a large corpus is a crucial step for tasks such as open-domain question answering and fact checking. Although neural retrieval outperforms traditi…
A Memory Efficient Baseline for Open Domain Question Answering
Gautier Izacard, Fabio Petroni, Lucas Hosseini +3
Recently, retrieval systems based on dense representations have led to important improvements in open-domain question answering, and related tasks. While very effective, this appro…
Generating Fact Checking Briefs
Angela Fan, Aleksandra Piktus, Fabio Petroni +5
Fact checking at scale is difficult -- while the number of active fact checking websites is growing, it remains too small for the needs of the contemporary media ecosystem. However…
Autoregressive Entity Retrieval
Nicola De Cao, Gautier Izacard, Sebastian Riedel +1
Entities are at the center of how we represent and aggregate knowledge. For instance, Encyclopedias such as Wikipedia are structured by entities (e.g., one per Wikipedia article).…
KILT: a Benchmark for Knowledge Intensive Language Tasks
Fabio Petroni, Aleksandra Piktus, Angela Fan +10
Challenging problems such as open-domain question answering, fact checking, slot filling and entity linking require access to large, external knowledge sources. While some models d…
How Context Affects Language Models' Factual Predictions
Fabio Petroni, Patrick Lewis, Aleksandra Piktus +4
When pre-trained on large unsupervised textual corpora, language models are able to store and retrieve factual knowledge to some extent, making it possible to use them directly for…
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Patrick Lewis, Ethan Perez, Aleksandra Piktus +9
Large pre-trained language models have been shown to store factual knowledge in their parameters, and achieve state-of-the-art results when fine-tuned on downstream NLP tasks. Howe…
Scalable Zero-shot Entity Linking with Dense Entity Retrieval
Ledell Wu, Fabio Petroni, Martin Josifoski +2
This paper introduces a conceptually simple, scalable, and highly effective BERT-based entity linking model, along with an extensive evaluation of its accuracy-speed trade-off. We…
How Decoding Strategies Affect the Verifiability of Generated Text
Luca Massarelli, Fabio Petroni, Aleksandra Piktus +5
Recent progress in pre-trained language models led to systems that are able to generate text of an increasingly high quality. While several works have investigated the fluency and…
Language Models as Knowledge Bases?
Fabio Petroni, Tim Rocktäschel, Patrick Lewis +4
Recent progress in pretraining language models on large textual corpora led to a surge of improvements for downstream NLP tasks. Whilst learning linguistic knowledge, these models…