interpretability 1knowledge retrieval 1language model pretraining 1open-domain question answering 1retrieval-augmented models 1
From the 1 of 2 linked papers with an AI index.
521 citations · 559 across the 2 of their papers we have counts for
2 papers
cs.CL2020★ 521 cited
REALM: Retrieval-Augmented Language Model Pre-Training
Kelvin Guu, Kenton Lee, Zora Tung +2
The paper introduces REALM, a language model that retrieves relevant documents from a large corpus during pre‑training and inference, enabling it to use external knowledge for task…
cs.CL2017★ 38 cited
SyntaxNet Models for the CoNLL 2017 Shared Task
Chris Alberti, Daniel Andor, Ivan Bogatyy +10
We describe a baseline dependency parsing system for the CoNLL2017 Shared Task. This system, which we call "ParseySaurus," uses the DRAGNN framework [Kong et al, 2017] to combine t…