8.9k citations · 11.5k across the 56 of their papers we have counts for
4 papers · 1 filter
Improving language models by retrieving from trillions of tokens
Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann +25
We enhance auto-regressive language models by conditioning on document chunks retrieved from a large corpus, based on local similarity with preceding tokens. With a trillion to…
Model-Value Inconsistency as a Signal for Epistemic Uncertainty
Angelos Filos, Eszter Vértes, Zita Marinho +7
Using a model of the environment and a value function, an agent can construct many estimates of a state's value, by unrolling the model for different lengths and bootstrapping with…
Top-KAST: Top-K Always Sparse Training
Siddhant M. Jayakumar, Razvan Pascanu, Jack W. Rae +2
Sparse neural networks are becoming increasingly important as the field seeks to improve the performance of existing models by scaling them up, while simultaneously trying to reduc…
Generative Art Using Neural Visual Grammars and Dual Encoders
Chrisantha Fernando, S. M. Ali Eslami, Jean-Baptiste Alayrac +3
Whilst there are perhaps only a few scientific methods, there seem to be almost as many artistic methods as there are artists. Artistic processes appear to inhabit the highest orde…