114 citations · 353 across the 89 of their papers we have counts for
49 papers · 1 filter
Inverting the Bellman Equation: From -Values to World Models
Alistair Letcher, Mattie Fellows, Alexander D. Goldie +3
Model-based and model-free reinforcement learning are traditionally viewed as separate paradigms: instead of learning a model of the transition kernel , model-free agents typica…
Goal-Conditioned Agents that Learn Everything All at Once
Michael Matthews, Matthew Jackson, Michael Beukman +5
A goal-conditioned reinforcement learning agent exploring an environment will see a wealth of information throughout a trajectory, most of which is discarded when only performing o…
Abstraction for Offline Goal-Conditioned Reinforcement Learning
Clarisse Wibault, Alexander Goldie, Antonio Villares +2
Markov Decision Processes (MDPs) often exhibit significant redundancy due to symmetries and shared structure across state-goal pairs in real-world Goal-Conditioned Reinforcement Le…
DiscoGen: Procedural Generation of Algorithm Discovery Tasks in Machine Learning
Alexander D. Goldie, Zilin Wang, Adrian Hayler +17
Automating the development of machine learning algorithms has the potential to unlock new breakthroughs. However, our ability to improve and evaluate algorithm discovery systems ha…
Measuring Progress in Reasoning Toward Mathematical Discovery with Automatic Verification
Erik Y. Wang, Sumeet R. Motwani, James V. Roggeveen +9
Can AI make progress on important, unsolved mathematical problems? Large language models are now capable of sophisticated mathematical and scientific reasoning, but whether they ca…
Preventing Learning Stagnation in PPO by Scaling to 1 Million Parallel Environments
Michael Beukman, Khimya Khetarpal, Zeyu Zheng +4
An agent's performance stagnating at a suboptimal level is a common problem in deep on-policy RL. Focusing on PPO, we show that plateaus in certain regimes arise not because of kno…
High entropy leads to symmetry-equivariant policies in Dec-POMDPs
Johannes Forkel, Constantin Ruhdorfer, Michael Beukman +2
We prove that in any Dec-POMDP, sufficiently high entropy regularization ensures that the policy gradient flow with tabular softmax parametrization always converges, for any initia…
Fully Offline Reinforcement Learning
Mattie Fellows, Clarisse Wibault, Uljad Berdica +3
Offline RL (ORL) promises safe and sample-efficient deployment but existing methods rely on undocumented online interactions for hyperparameter tuning and lack reliable fully offli…
No Regrets: Investigating and Improving Regret Approximations for Curriculum Discovery
Alexander Rutherford, Michael Beukman, Timon Willi +3
What data or environments to use for training to improve downstream performance is a longstanding and very topical question in reinforcement learning. In particular, Unsupervised E…
Can Learned Optimization Make Reinforcement Learning Less Difficult?
Alexander David Goldie, Chris Lu, Matthew Thomas Jackson +2
While reinforcement learning (RL) holds great potential for decision making in the real world, it suffers from a number of unique difficulties which often need specific considerati…
Simplifying Deep Temporal Difference Learning
Matteo Gallici, Mattie Fellows, Benjamin Ellis +4
Q-learning played a foundational role in the field reinforcement learning (RL). However, TD algorithms with off-policy data, such as Q-learning, or nonlinear function approximation…
Mixture of Experts in a Mixture of RL settings
Timon Willi, Johan Obando-Ceron, Jakob Foerster +2
Mixtures of Experts (MoEs) have gained prominence in (self-)supervised learning due to their enhanced inference efficiency, adaptability to distributed training, and modularity. Pr…
Behaviour Distillation
Andrei Lupu, Chris Lu, Jarek Liesen +2
Dataset distillation aims to condense large datasets into a small number of synthetic examples that can be used as drop-in replacements when training new models. It has application…
Discovering Minimal Reinforcement Learning Environments
Jarek Liesen, Chris Lu, Andrei Lupu +3
Reinforcement learning (RL) agents are commonly trained and evaluated in the same environment. In contrast, humans often train in a specialized environment before being evaluated,…
HelloFresh: LLM Evaluations on Streams of Real-World Human Editorial Actions across X Community Notes and Wikipedia edits
Tim Franzmeyer, Aleksandar Shtedritski, Samuel Albanie +3
Benchmarks have been essential for driving progress in machine learning. A better understanding of LLM capabilities on real world tasks is vital for safe development. Designing ade…
Discovering Preference Optimization Algorithms with and for Large Language Models
Chris Lu, Samuel Holt, Claudio Fanconi +4
Offline preference optimization is a key method for enhancing and controlling the quality of Large Language Model (LLM) outputs. Typically, preference optimization is approached as…
Risks and Opportunities of Open-Source Generative AI
Francisco Eiras, Aleksandar Petrov, Bertie Vidgen +22
Applications of Generative AI (Gen AI) are expected to revolutionize a number of different areas, ranging from science & medicine to education. The potential for these seismic chan…
Near to Mid-term Risks and Opportunities of Open-Source Generative AI
Francisco Eiras, Aleksandar Petrov, Bertie Vidgen +21
In the next few years, applications of Generative AI are expected to revolutionize a number of different areas, ranging from science & medicine to education. The potential for thes…
Rethinking Out-of-Distribution Detection for Reinforcement Learning: Advancing Methods for Evaluation and Detection
Linas Nasvytis, Kai Sandbrink, Jakob Foerster +2
While reinforcement learning (RL) algorithms have been successfully applied across numerous sequential decision-making problems, their generalization to unforeseen testing environm…
Policy-Guided Diffusion
Matthew Thomas Jackson, Michael Tryfan Matthews, Cong Lu +3
In many real-world settings, agents must learn from an offline dataset gathered by some prior behavior policy. Such a setting naturally leads to distribution shift between the beha…
Foundational Challenges in Assuring Alignment and Safety of Large Language Models
Usman Anwar, Abulhair Saparov, Javier Rando +39
This work identifies 18 foundational challenges in assuring the alignment and safety of large language models (LLMs). These challenges are organized into three different categories…
JaxUED: A simple and useable UED library in Jax
Samuel Coward, Michael Beukman, Jakob Foerster
We present JaxUED, an open-source library providing minimal dependency implementations of modern Unsupervised Environment Design (UED) algorithms in Jax. JaxUED leverages hardware…
Discovering Temporally-Aware Reinforcement Learning Algorithms
Matthew Thomas Jackson, Chris Lu, Louis Kirsch +3
Recent advancements in meta-learning have enabled the automatic discovery of novel reinforcement learning algorithms parameterized by surrogate objective functions. To improve upon…
Analysing the Sample Complexity of Opponent Shaping
Kitty Fung, Qizhen Zhang, Chris Lu +3
Learning in general-sum games often yields collectively sub-optimal results. Addressing this, opponent shaping (OS) methods actively guide the learning processes of other agents, e…