6.4k citations · 7.6k across the 17 of their papers we have counts for
18 papers
Fine-Tuned In-Context Learners for Efficient Adaptation
Jorg Bornschein, Clare Lyle, Yazhe Li +3
When adapting large language models (LLMs) to a specific downstream task, two primary approaches are commonly employed: (1) prompt engineering, often with in-context few-shot learn…
On the Hardness of Conditional Independence Testing In Practice
Zheng He, Roman Pogodin, Yazhe Li +3
Tests of conditional independence (CI) underpin a number of important problems in machine learning and statistics, from causal discovery to evaluation of predictor fairness and out…
Transformers for Supervised Online Continual Learning
Jorg Bornschein, Yazhe Li, Amal Rannen-Triki
Transformers have become the dominant architecture for sequence modeling tasks such as natural language processing or audio processing, and they are now even considered for tasks t…
Denoising Autoregressive Representation Learning
Yazhe Li, Jorg Bornschein, Ting Chen
In this paper, we explore a new generative approach for learning visual representations. Our method, DARL, employs a decoder-only Transformer to predict image patches autoregressiv…
Practical Kernel Tests of Conditional Independence
Roman Pogodin, Antonin Schrab, Yazhe Li +2
We describe a data-efficient, kernel-based approach to statistical testing of conditional independence. A major challenge of conditional independence testing is to obtain the corre…
Investigating the role of model-based learning in exploration and transfer
Jacob Walker, Eszter Vértes, Yazhe Li +4
State of the art reinforcement learning has enabled training agents on tasks of ever increasing complexity. However, the current paradigm tends to favor training agents from scratc…
Evaluating Representations with Readout Model Switching
Yazhe Li, Jorg Bornschein, Marcus Hutter
Although much of the success of Deep Learning builds on learning good representations, a rigorous method to evaluate their quality is lacking. In this paper, we treat the evaluatio…
Efficient Conditionally Invariant Representation Learning
Roman Pogodin, Namrata Deka, Yazhe Li +3
We introduce the Conditional Independence Regression CovariancE (CIRCE), a measure of conditional independence for multivariate continuous-valued variables. CIRCE applies as a regu…
Where Should I Spend My FLOPS? Efficiency Evaluations of Visual Pre-training Methods
Skanda Koppula, Yazhe Li, Evan Shelhamer +5
Self-supervised methods have achieved remarkable success in transfer learning, often achieving the same or better accuracy than supervised pre-training. Most prior work has done so…
Sequential Learning Of Neural Networks for Prequential MDL
Jorg Bornschein, Yazhe Li, Marcus Hutter
Minimum Description Length (MDL) provides a framework and an objective for principled model evaluation. It formalizes Occam's Razor and can be applied to data from non-stationary s…
Procedural Generalization by Planning with Self-Supervised World Models
Ankesh Anand, Jacob Walker, Yazhe Li +5
One of the key promises of model-based reinforcement learning is the ability to generalize using an internal model of the world to make predictions in novel environments and tasks.…
Vector Quantized Models for Planning
Sherjil Ozair, Yazhe Li, Ali Razavi +3
Recent developments in the field of model-based RL have proven successful in a range of environments, especially ones where planning is essential. However, such successes have been…
Self-Supervised Learning with Kernel Dependence Maximization
Yazhe Li, Roman Pogodin, Danica J. Sutherland +1
We approach self-supervised learning of image representations from a statistical dependence perspective, proposing Self-Supervised Learning with the Hilbert-Schmidt Independence Cr…
Low Bit-Rate Speech Coding with VQ-VAE and a WaveNet Decoder
Cristina Gârbacea, Aäron van den Oord, Yazhe Li +4
In order to efficiently transmit and store speech signals, speech codecs create a minimally redundant representation of the input signal which is then decoded at the receiver with…
Representation Learning with Contrastive Predictive Coding
Aaron van den Oord, Yazhe Li, Oriol Vinyals
While supervised learning has enabled great progress in many applications, unsupervised learning has not seen such widespread adoption, and remains an important and challenging end…
DeepMind Control Suite
Yuval Tassa, Yotam Doron, Alistair Muldal +9
The DeepMind Control Suite is a set of continuous control tasks with a standardised structure and interpretable rewards, intended to serve as performance benchmarks for reinforceme…
Parallel WaveNet: Fast High-Fidelity Speech Synthesis
Aaron van den Oord, Yazhe Li, Igor Babuschkin +19
The recently-developed WaveNet architecture is the current state of the art in realistic speech synthesis, consistently rated as more natural sounding for many different languages…
Model-Free Episodic Control
Charles Blundell, Benigno Uria, Alexander Pritzel +6
State of the art deep reinforcement learning algorithms take many millions of interactions to attain human-level performance. Humans, on the other hand, can very quickly exploit hi…