291 citations · 385 across the 16 of their papers we have counts for
16 papers
Revisiting Dynamic Evaluation: Online Adaptation for Large Language Models
Amal Rannen-Triki, Jorg Bornschein, Razvan Pascanu +5
We consider the problem of online fine tuning the parameters of a language model at test time, also known as dynamic evaluation. While it is generally known that this approach impr…
Denoising Autoregressive Representation Learning
Yazhe Li, Jorg Bornschein, Ting Chen
In this paper, we explore a new generative approach for learning visual representations. Our method, DARL, employs a decoder-only Transformer to predict image patches autoregressiv…
Towards Robust and Efficient Continual Language Learning
Adam Fisch, Amal Rannen-Triki, Razvan Pascanu +4
As the application space of language models continues to evolve, a natural question to ask is how we can quickly adapt models to new tasks. We approach this classic question from a…
Kalman Filter for Online Classification of Non-Stationary Data
Michalis K. Titsias, Alexandre Galashov, Amal Rannen-Triki +3
In Online Continual Learning (OCL) a learning system receives a stream of data and sequentially performs prediction and training steps. Important challenges in OCL are concerned wi…
DiscoGen: Learning to Discover Gene Regulatory Networks
Nan Rosemary Ke, Sara-Jane Dunn, Jorg Bornschein +11
Accurately inferring Gene Regulatory Networks (GRNs) is a critical and challenging task in biology. GRNs model the activatory and inhibitory interactions between genes and are inhe…
Evaluating Representations with Readout Model Switching
Yazhe Li, Jorg Bornschein, Marcus Hutter
Although much of the success of Deep Learning builds on learning good representations, a rigorous method to evaluate their quality is lacking. In this paper, we treat the evaluatio…