89 citations · 139 across the 9 of their papers we have counts for
12 papers
GLIMMER: generalized late-interaction memory reranker
Michiel de Jong, Yury Zemlyanskiy, Nicholas FitzGerald +3
Memory-augmentation is a powerful approach for efficiently incorporating external information into language models, but leads to reduced performance relative to retrieving text. Re…
Pre-computed memory or on-the-fly encoding? A hybrid approach to retrieval augmentation makes the most of your compute
Michiel de Jong, Yury Zemlyanskiy, Nicholas FitzGerald +4
Retrieval-augmented language models such as Fusion-in-Decoder are powerful, setting the state of the art on a variety of knowledge-intensive tasks. However, they are also expensive…
FiDO: Fusion-in-Decoder optimized for stronger performance and faster inference
Michiel de Jong, Yury Zemlyanskiy, Joshua Ainslie +4
Fusion-in-Decoder (FiD) is a powerful retrieval-augmented language model that sets the state-of-the-art on many knowledge-intensive NLP tasks. However, the architecture used for Fi…
Mention Memory: incorporating textual knowledge into Transformers through entity mention attention
Michiel de Jong, Yury Zemlyanskiy, Nicholas FitzGerald +2
Natural language understanding tasks such as open-domain question answering often require retrieving and assimilating factual information from multiple sources. We propose to addre…
MOLEMAN: Mention-Only Linking of Entities with a Mention Annotation Network
Nicholas FitzGerald, Jan A. Botha, Daniel Gillick +3
We present an instance-based nearest neighbor approach to entity linking. In contrast to most prior entity retrieval systems which represent each entity with a single vector, we bu…
Empirical Evaluation of Pretraining Strategies for Supervised Entity Linking
Thibault Févry, Nicholas FitzGerald, Livio Baldini Soares +1
In this work, we present an entity linking model which combines a Transformer architecture with large scale pretraining from Wikipedia links. Our model achieves the state-of-the-ar…
Entities as Experts: Sparse Memory Access with Entity Supervision
Thibault Févry, Livio Baldini Soares, Nicholas FitzGerald +2
We focus on the problem of capturing declarative knowledge about entities in the learned parameters of a language model. We introduce a new model - Entities as Experts (EAE) - that…
Learning Cross-Context Entity Representations from Text
Jeffrey Ling, Nicholas FitzGerald, Zifei Shan +4
Language modeling tasks, in which words, or word-pieces, are predicted on the basis of a local context, have been very effective for learning word embeddings and context dependent…
Matching the Blanks: Distributional Similarity for Relation Learning
Livio Baldini Soares, Nicholas FitzGerald, Jeffrey Ling +1
General purpose relation extractors, which can model arbitrary relations, are a core aspiration in information extraction. Efforts have been made to build general purpose extractor…
Large-Scale QA-SRL Parsing
Nicholas FitzGerald, Julian Michael, Luheng He +1
We present a new large-scale corpus of Question-Answer driven Semantic Role Labeling (QA-SRL) annotations, and the first high-quality QA-SRL parser. Our corpus, QA-SRL Bank 2.0, co…
Long Short-Term Memory as a Dynamically Computed Element-wise Weighted Sum
Omer Levy, Kenton Lee, Nicholas FitzGerald +1
LSTMs were introduced to combat vanishing gradients in simple RNNs by augmenting them with gated additive recurrent connections. We present an alternative view to explain the succe…
A Joint Model of Language and Perception for Grounded Attribute Learning
Cynthia Matuszek, Nicholas FitzGerald, Luke Zettlemoyer +2
As robots become more ubiquitous and capable, it becomes ever more important to enable untrained users to easily interact with them. Recently, this has led to study of the language…