14.1k citations · 89.2k across the 108 of their papers we have counts for
9 papers · 1 filter
Improving language models by retrieving from trillions of tokens
Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann +25
We enhance auto-regressive language models by conditioning on document chunks retrieved from a large corpus, based on local similarity with preceding tokens. With a trillion to…
WikiGraphs: A Wikipedia Text - Knowledge Graph Paired Dataset
Luyu Wang, Yujia Li, Ozlem Aslan +1
We present a new dataset of Wikipedia articles each paired with a knowledge graph, to facilitate the research in conditional text generation, graph generation and graph representat…
The Benchmark Lottery
Mostafa Dehghani, Yi Tay, Alexey A. Gritsenko +5
The world of empirical machine learning (ML) strongly relies on benchmarks in order to determine the relative effectiveness of different algorithms and methods. This paper proposes…
Multimodal Few-Shot Learning with Frozen Language Models
Maria Tsimpoukelli, Jacob Menick, Serkan Cabi +3
When trained at sufficient scale, auto-regressive language models exhibit the notable ability to learn a new language task after being prompted with just a few examples. Here, we p…
Perceiver IO: A General Architecture for Structured Inputs & Outputs
Andrew Jaegle, Sebastian Borgeaud, Jean-Baptiste Alayrac +12
A central goal of machine learning is the development of systems that can solve many problems in as many data domains as possible. Current architectures, however, cannot be applied…
Vector Quantized Models for Planning
Sherjil Ozair, Yazhe Li, Ali Razavi +3
Recent developments in the field of model-based RL have proven successful in a range of environments, especially ones where planning is essential. However, such successes have been…