1.1k citations · 5k across the 117 of their papers we have counts for
86 papers · 1 filter
DiscoGen: Procedural Generation of Algorithm Discovery Tasks in Machine Learning
Alexander D. Goldie, Zilin Wang, Adrian Hayler +17
Automating the development of machine learning algorithms has the potential to unlock new breakthroughs. However, our ability to improve and evaluate algorithm discovery systems ha…
Evolution Strategies at the Hyperscale
Bidipta Sarkar, Mattie Fellows, Juan Agustin Duque +17
Evolution Strategies (ES) is a class of powerful black-box optimisation methods that are highly parallelisable and can handle non-differentiable and noisy objectives. However, naïv…
How Should We Meta-Learn Reinforcement Learning Algorithms?
Alexander David Goldie, Zilin Wang, Jaron Cohen +2
The process of meta-learning algorithms from data, instead of relying on manual design, is growing in popularity as a paradigm for improving the performance of machine learning sys…
GoalLadder: Incremental Goal Discovery with Vision-Language Models
Alexey Zakharov, Shimon Whiteson
Natural language can offer a concise and human-interpretable means of specifying reinforcement learning (RL) tasks. The ability to extract rewards from a language instruction can e…
A Clean Slate for Offline Reinforcement Learning
Matthew Thomas Jackson, Uljad Berdica, Jarek Liesen +2
Progress in offline reinforcement learning (RL) has been impeded by ambiguous problem definitions and entangled algorithmic designs, resulting in inconsistent implementations, insu…
A Survey of In-Context Reinforcement Learning
Amir Moeini, Jiuqi Wang, Jacob Beck +4
Reinforcement learning (RL) agents typically optimize their policies by performing expensive backward passes to update their network parameters. However, some agents can solve new…
Adam on Local Time: Addressing Nonstationarity in RL with Relative Adam Timesteps
Benjamin Ellis, Matthew T. Jackson, Andrei Lupu +4
In reinforcement learning (RL), it is common to apply techniques used broadly in machine learning such as neural network function approximators and momentum-based optimizers. Howev…
Can Learned Optimization Make Reinforcement Learning Less Difficult?
Alexander David Goldie, Chris Lu, Matthew Thomas Jackson +2
While reinforcement learning (RL) holds great potential for decision making in the real world, it suffers from a number of unique difficulties which often need specific considerati…
A Bayesian Solution To The Imitation Gap
Risto Vuorio, Mattie Fellows, Cong Lu +2
In many real-world settings, an agent must learn to act in environments where no reward signal can be specified, but a set of expert demonstrations is available. Imitation learning…
SplAgger: Split Aggregation for Meta-Reinforcement Learning
Jacob Beck, Matthew Jackson, Risto Vuorio +2
A core ambition of reinforcement learning (RL) is the creation of agents capable of rapid learning in novel tasks. Meta-RL aims to achieve this by directly learning such agents. Bl…
Discovering Temporally-Aware Reinforcement Learning Algorithms
Matthew Thomas Jackson, Chris Lu, Louis Kirsch +3
Recent advancements in meta-learning have enabled the automatic discovery of novel reinforcement learning algorithms parameterized by surrogate objective functions. To improve upon…
Distilling Morphology-Conditioned Hypernetworks for Efficient Universal Morphology Control
Zheng Xiong, Risto Vuorio, Jacob Beck +3
Learning a universal policy across different robot morphologies can significantly improve learning efficiency and enable zero-shot generalization to unseen morphologies. However, l…
JaxMARL: Multi-Agent RL Environments and Algorithms in JAX
Alexander Rutherford, Benjamin Ellis, Matteo Gallici +18
Benchmarks are crucial in the development of machine learning algorithms, with available environments significantly influencing reinforcement learning (RL) research. Traditionally,…
Discovering General Reinforcement Learning Algorithms with Adversarial Environment Design
Matthew Thomas Jackson, Minqi Jiang, Jack Parker-Holder +5
The past decade has seen vast progress in deep reinforcement learning (RL) on the back of algorithms manually designed by human researchers. Recently, it has been shown that it is…
Hierarchical Imitation Learning for Stochastic Environments
Maximilian Igl, Punit Shah, Paul Mougin +5
Many applications of imitation learning require the agent to generate the full distribution of behaviour observed in the training data. For example, to evaluate the safety of auton…
Recurrent Hypernetworks are Surprisingly Strong in Meta-RL
Jacob Beck, Risto Vuorio, Zheng Xiong +1
Deep reinforcement learning (RL) is notoriously impractical to deploy due to sample inefficiency. Meta-RL directly addresses this sample inefficiency by learning to perform few-sho…
Bayesian Exploration Networks
Mattie Fellows, Brandon Kaplowitz, Christian Schroeder de Witt +1
Bayesian reinforcement learning (RL) offers a principled and elegant approach for sequential decision making under uncertainty. Most notably, Bayesian agents do not face an explora…
Trust-Region-Free Policy Optimization for Stochastic Policies
Mingfei Sun, Benjamin Ellis, Anuj Mahajan +3
Trust Region Policy Optimization (TRPO) is an iterative method that simultaneously maximizes a surrogate objective and enforces a trust region constraint over consecutive policies…
Why Target Networks Stabilise Temporal Difference Methods
Mattie Fellows, Matthew J. A. Smith, Shimon Whiteson
Integral to recent successes in deep reinforcement learning has been a class of temporal difference methods that use infrequently updated target values for policy evaluation in a M…
A Tutorial on Meta-Reinforcement Learning
Jacob Beck, Risto Vuorio, Evan Zheran Liu +4
While deep reinforcement learning (RL) has fueled multiple high-profile successes in machine learning, it is held back from more widespread adoption by its often poor data efficien…
SMACv2: An Improved Benchmark for Cooperative Multi-Agent Reinforcement Learning
Benjamin Ellis, Jonathan Cook, Skander Moalla +5
The availability of challenging benchmarks has played a key role in the recent progress of machine learning. In cooperative multi-agent reinforcement learning, the StarCraft Multi-…
Hypernetworks in Meta-Reinforcement Learning
Jacob Beck, Matthew Thomas Jackson, Risto Vuorio +1
Training a reinforcement learning (RL) agent on a real-world robotics task remains generally impractical due to sample inefficiency. Multi-task RL and meta-RL aim to improve sample…
Equivariant Networks for Zero-Shot Coordination
Darius Muglich, Christian Schroeder de Witt, Elise van der Pol +2
Successful coordination in Dec-POMDPs requires agents to adopt robust strategies and interpretable styles of play for their partner. A common failure mode is symmetry breaking, whe…
An Investigation of the Bias-Variance Tradeoff in Meta-Gradients
Risto Vuorio, Jacob Beck, Shimon Whiteson +2
Meta-gradients provide a general approach for optimizing the meta-parameters of reinforcement learning (RL) algorithms. Estimation of meta-gradients is central to the performance o…