6.4k citations · 7.6k across the 17 of their papers we have counts for
12 papers · 1 filter
Fine-Tuned In-Context Learners for Efficient Adaptation
Jorg Bornschein, Clare Lyle, Yazhe Li +3
When adapting large language models (LLMs) to a specific downstream task, two primary approaches are commonly employed: (1) prompt engineering, often with in-context few-shot learn…
Transformers for Supervised Online Continual Learning
Jorg Bornschein, Yazhe Li, Amal Rannen-Triki
Transformers have become the dominant architecture for sequence modeling tasks such as natural language processing or audio processing, and they are now even considered for tasks t…
Denoising Autoregressive Representation Learning
Yazhe Li, Jorg Bornschein, Ting Chen
In this paper, we explore a new generative approach for learning visual representations. Our method, DARL, employs a decoder-only Transformer to predict image patches autoregressiv…
Practical Kernel Tests of Conditional Independence
Roman Pogodin, Antonin Schrab, Yazhe Li +2
We describe a data-efficient, kernel-based approach to statistical testing of conditional independence. A major challenge of conditional independence testing is to obtain the corre…
Investigating the role of model-based learning in exploration and transfer
Jacob Walker, Eszter Vértes, Yazhe Li +4
State of the art reinforcement learning has enabled training agents on tasks of ever increasing complexity. However, the current paradigm tends to favor training agents from scratc…
Evaluating Representations with Readout Model Switching
Yazhe Li, Jorg Bornschein, Marcus Hutter
Although much of the success of Deep Learning builds on learning good representations, a rigorous method to evaluate their quality is lacking. In this paper, we treat the evaluatio…
Efficient Conditionally Invariant Representation Learning
Roman Pogodin, Namrata Deka, Yazhe Li +3
We introduce the Conditional Independence Regression CovariancE (CIRCE), a measure of conditional independence for multivariate continuous-valued variables. CIRCE applies as a regu…
Procedural Generalization by Planning with Self-Supervised World Models
Ankesh Anand, Jacob Walker, Yazhe Li +5
One of the key promises of model-based reinforcement learning is the ability to generalize using an internal model of the world to make predictions in novel environments and tasks.…
Vector Quantized Models for Planning
Sherjil Ozair, Yazhe Li, Ali Razavi +3
Recent developments in the field of model-based RL have proven successful in a range of environments, especially ones where planning is essential. However, such successes have been…
Low Bit-Rate Speech Coding with VQ-VAE and a WaveNet Decoder
Cristina Gârbacea, Aäron van den Oord, Yazhe Li +4
In order to efficiently transmit and store speech signals, speech codecs create a minimally redundant representation of the input signal which is then decoded at the receiver with…
Representation Learning with Contrastive Predictive Coding
Aaron van den Oord, Yazhe Li, Oriol Vinyals
While supervised learning has enabled great progress in many applications, unsupervised learning has not seen such widespread adoption, and remains an important and challenging end…
Parallel WaveNet: Fast High-Fidelity Speech Synthesis
Aaron van den Oord, Yazhe Li, Igor Babuschkin +19
The recently-developed WaveNet architecture is the current state of the art in realistic speech synthesis, consistently rated as more natural sounding for many different languages…