137 citations · 527 across the 41 of their papers we have counts for
46 papers
At the Limits of Computability: Steps Towards Autonomous AI Discovery on the Busy Beaver Frontier
Marco Dos Santos, Vivek Veeriah, Alex Havrilla +4
Recent breakthroughs in AI for mathematics---ranging from Olympiad medal performances to resolving long-standing open conjectures---have rapidly expanded what automated systems can…
Conditional Generation of Creative Chess Puzzles with Diffusion Models
Aatu Selkee, Severi Rissanen, Xidong Feng +2
While modern language models demonstrate impressive generative capabilities, they often struggle with constrained, counter-intuitive creative tasks. To address this limitation, we…
COrigami: An AI Pipeline for Co-Designing Flat-Foldable Visually Recognisable Origami
Tom Zahavy, Shaobo Hou, Thomas Tumiel +16
While generative AI has achieved remarkable success in solving problems with verifiable solutions, generating physical art that satisfies both strict geometric constraints and subj…
Generating Creative Chess Puzzles
Xidong Feng, Vivek Veeriah, Marcus Chiam +10
While Generative AI rapidly advances in various domains, generating truly creative, aesthetic, and counter-intuitive outputs remains a challenge. This paper presents an approach to…
Evaluating In Silico Creativity: An Expert Review of AI Chess Compositions
Vivek Veeriah, Federico Barbero, Marcus Chiam +10
The rapid advancement of Generative AI has raised significant questions regarding its ability to produce creative and novel outputs. Our recent work investigates this question with…
Mastering Board Games by External and Internal Planning with Language Models
John Schultz, Jakub Adamek, Matej Jusup +13
Advancing planning and reasoning capabilities of Large Language Models (LLMs) is one of the key prerequisites towards unlocking their potential for performing reliably in complex a…
APART: Diverse Skill Discovery using All Pairs with Ascending Reward and DropouT
Hadar Schreiber Galler, Tom Zahavy, Guillaume Desjardins +1
We study diverse skill discovery in reward-free environments, aiming to discover all possible skills in simple grid-world environments where prior methods have struggled to succeed…
Diversifying AI: Towards Creative Chess with AlphaZero
Tom Zahavy, Vivek Veeriah, Shaobo Hou +7
In recent years, Artificial Intelligence (AI) systems have surpassed human intelligence in a variety of computational tasks. However, AI systems, like humans, make mistakes, have b…
Acceleration in Policy Optimization
Veronica Chelu, Tom Zahavy, Arthur Guez +2
We work towards a unifying paradigm for accelerating policy optimization methods in reinforcement learning (RL) by integrating foresight in the policy improvement step via optimist…
Discovering Attention-Based Genetic Algorithms via Meta-Black-Box Optimization
Robert Tjarko Lange, Tom Schaul, Yutian Chen +4
Genetic algorithms constitute a family of black-box optimization algorithms, which take inspiration from the principles of biological evolution. While they provide a general-purpos…
ReLOAD: Reinforcement Learning with Optimistic Ascent-Descent for Last-Iterate Convergence in Constrained MDPs
Ted Moskovitz, Brendan O'Donoghue, Vivek Veeriah +3
In recent years, Reinforcement Learning (RL) has been applied to real-world problems with increasing success. Such applications often require to put constraints on the agent's beha…
Optimistic Meta-Gradients
Sebastian Flennerhag, Tom Zahavy, Brendan O'Donoghue +3
We study the connection between gradient-based meta-learning and convex op-timisation. We observe that gradient descent with momentum is a special case of meta-gradients, and build…
POMRL: No-Regret Learning-to-Plan with Increasing Horizons
Khimya Khetarpal, Claire Vernade, Brendan O'Donoghue +2
We study the problem of planning under model uncertainty in an online meta-reinforcement learning (RL) setting where an agent is presented with a sequence of related tasks with lim…
Discovering Evolution Strategies via Meta-Black-Box Optimization
Robert Tjarko Lange, Tom Schaul, Yutian Chen +5
Optimizing functions without access to gradients is the remit of black-box methods such as evolution strategies. While highly general, their learning dynamics are often times heuri…
Palm up: Playing in the Latent Manifold for Unsupervised Pretraining
Hao Liu, Tom Zahavy, Volodymyr Mnih +1
Large and diverse datasets have been the cornerstones of many impressive advancements in artificial intelligence. Intelligent creatures, however, learn by interacting with the envi…
Meta-Gradients in Non-Stationary Environments
Jelena Luketina, Sebastian Flennerhag, Yannick Schroecker +3
Meta-gradient methods (Xu et al., 2018; Zahavy et al., 2020) offer a promising solution to the problem of hyperparameter selection and adaptation in non-stationary reinforcement le…
Discovering Policies with DOMiNO: Diversity Optimization Maintaining Near Optimality
Tom Zahavy, Yannick Schroecker, Feryal Behbahani +4
Finding different solutions to the same problem is a key aspect of intelligence associated with creativity and adaptation to novel situations. In reinforcement learning, a set of d…
Bootstrapped Meta-Learning
Sebastian Flennerhag, Yannick Schroecker, Tom Zahavy +3
Meta-learning empowers artificial intelligence to increase its efficiency by learning how to learn. Unlocking this potential involves overcoming a challenging meta-optimisation pro…
Emphatic Algorithms for Deep Reinforcement Learning
Ray Jiang, Tom Zahavy, Zhongwen Xu +4
Off-policy learning allows us to learn about possible policies of behavior from experience generated by a different behavior policy. Temporal difference (TD) learning algorithms ca…
Discovering Diverse Nearly Optimal Policies with Successor Features
Tom Zahavy, Brendan O'Donoghue, Andre Barreto +3
Finding different solutions to the same problem is a key aspect of intelligence associated with creativity and adaptation to novel situations. In reinforcement learning, a set of d…
Reward is enough for convex MDPs
Tom Zahavy, Brendan O'Donoghue, Guillaume Desjardins +1
Maximising a cumulative reward function that is Markov and stationary, i.e., defined over state-action pairs and independent of time, is sufficient to capture many kinds of goals i…
Discovery of Options via Meta-Learned Subgoals
Vivek Veeriah, Tom Zahavy, Matteo Hessel +6
Temporal abstractions in the form of options have been shown to help reinforcement learning (RL) agents learn faster. However, despite prior work on this topic, the problem of disc…
Online Apprenticeship Learning
Lior Shani, Tom Zahavy, Shie Mannor
In Apprenticeship Learning (AL), we are given a Markov Decision Process (MDP) without access to the cost function. Instead, we observe trajectories sampled by an expert that acts a…
Discovering a set of policies for the worst case reward
Tom Zahavy, Andre Barreto, Daniel J Mankowitz +4
We study the problem of how to construct a set of policies that can be composed together to solve a collection of reinforcement learning tasks. Each task is a different reward func…