19 citations · 37 across the 22 of their papers we have counts for
23 papers
Beyond Static Graph World Models: Learning Stochastic Latent Dynamics over Evolving Topologies
Alex Schutz, Nick Hawes, Victor-Alexandru Darvariu
Graph-based world models have recently emerged as a means of learning transitions over relational state representations. However, existing approaches are largely limited to fixed-t…
Improving Regret Approximation for Unsupervised Dynamic Environment Generation
Harry Mead, Bruno Lacerda, Jakob Foerster +1
Unsupervised Environment Design (UED) seeks to automatically generate training curricula for reinforcement learning (RL) agents, with the goal of improving generalisation and zero-…
JaxWildfire: A GPU-Accelerated Wildfire Simulator for Reinforcement Learning
Ufuk Çakır, Victor-Alexandru Darvariu, Bruno Lacerda +1
Artificial intelligence methods are increasingly being explored for managing wildfires and other natural hazards. In particular, reinforcement learning (RL) is a promising path tow…
Gaussian Process Aggregation for Root-Parallel Monte Carlo Tree Search with Continuous Actions
Junlin Xiao, Victor-Alexandru Darvariu, Bruno Lacerda +1
Monte Carlo Tree Search is a cornerstone algorithm for online planning, and its root-parallel variant is widely used when wall clock time is limited but best performance is desired…
Neural Value Iteration
Yang You, Ufuk Çakır, Alex Schutz +1
The value function of a POMDP exhibits the piecewise-linear-convex (PWLC) property and can be represented as a finite set of hyperplanes, known as -vectors. Most state-of-the-ar…
Tackling GNARLy Problems: Graph Neural Algorithmic Reasoning Reimagined through Reinforcement Learning
Alex Schutz, Victor-Alexandru Darvariu, Efimia Panagiotaki +2
Neural algorithmic reasoning (NAR) is a paradigm that trains neural networks to execute classic algorithms by supervised learning. Despite its successes, important limitations rema…
Scalable Solution Methods for Dec-POMDPs with Deterministic Dynamics
Yang You, Alex Schutz, Zhikun Li +3
Many high-level multi-agent planning problems, including multi-robot navigation and path planning, can be effectively modeled using deterministic actions and observations. In this…
Partially Observable Monte-Carlo Graph Search
Yang You, Vincent Thomas, Alex Schutz +3
Currently, large partially observable Markov decision processes (POMDPs) are often solved by sampling-based online methods which interleave planning and execution phases. However,…
A Finite-State Controller Based Offline Solver for Deterministic POMDPs
Alex Schutz, Yang You, Matias Mattamala +3
Deterministic partially observable Markov decision processes (DetPOMDPs) often arise in planning problems where the agent is uncertain about its environmental state but can act and…
Return Capping: Sample-Efficient CVaR Policy Gradient Optimisation
Harry Mead, Clarissa Costen, Bruno Lacerda +1
When optimising for conditional value at risk (CVaR) using policy gradients (PG), current methods rely on discarding a large proportion of trajectories, resulting in poor sample ef…
Ro-To-Go! Robust Reactive Control with Signal Temporal Logic
Roland Ilyes, Lara Brudermüller, Nick Hawes +1
Signal Temporal Logic (STL) robustness is a common objective for optimal robot control, but its dependence on history limits the robot's decision-making capabilities when used in M…
No Regrets: Investigating and Improving Regret Approximations for Curriculum Discovery
Alexander Rutherford, Michael Beukman, Timon Willi +3
What data or environments to use for training to improve downstream performance is a longstanding and very topical question in reinforcement learning. In particular, Unsupervised E…
Monte Carlo Tree Search with Boltzmann Exploration
Michael Painter, Mohamed Baioumy, Nick Hawes +1
Monte-Carlo Tree Search (MCTS) methods, such as Upper Confidence Bound applied to Trees (UCT), are instrumental to automated planning techniques. However, UCT can be slow to explor…
JaxMARL: Multi-Agent RL Environments and Algorithms in JAX
Alexander Rutherford, Benjamin Ellis, Matteo Gallici +18
Benchmarks are crucial in the development of machine learning algorithms, with available environments significantly influencing reinforcement learning (RL) research. Traditionally,…
DITTO: Offline Imitation Learning with World Models
Branton DeMoss, Paul Duckworth, Jakob Foerster +2
For imitation learning algorithms to scale to real-world challenges, they must handle high-dimensional observations, offline learning, and policy-induced covariate-shift. We propos…
One Risk to Rule Them All: A Risk-Sensitive Perspective on Model-Based Offline Reinforcement Learning
Marc Rigter, Bruno Lacerda, Nick Hawes
Offline reinforcement learning (RL) is suitable for safety-critical domains where online exploration is too costly or dangerous. In such safety-critical settings, decision-making s…
VP-STO: Via-point-based Stochastic Trajectory Optimization for Reactive Robot Behavior
Julius Jankowski, Lara Brudermüller, Nick Hawes +1
Achieving reactive robot behavior in complex dynamic environments is still challenging as it relies on being able to solve trajectory optimization problems quickly enough, such tha…
Unbiased Active Inference for Classical Control
Mohamed Baioumy, Corrado Pezzato, Riccardo Ferrari +1
Active inference is a mathematical framework that originated in computational neuroscience. Recently, it has been demonstrated as a promising approach for constructing goal-driven…
Beta Residuals: Improving Fault-Tolerant Control for Sensory Faults via Bayesian Inference and Precision Learning
Mohamed Baioumy, William Hartemink, Riccardo M. G. Ferrari +1
Model-based fault-tolerant control (FTC) often consists of two distinct steps: fault detection & isolation (FDI), and fault accommodation. In this work we investigate posing fault-…
RAMBO-RL: Robust Adversarial Model-Based Offline Reinforcement Learning
Marc Rigter, Bruno Lacerda, Nick Hawes
Offline reinforcement learning (RL) aims to find performant policies from logged data without further environment interaction. Model-based algorithms, which learn a model of the en…
Planning for Risk-Aversion and Expected Value in MDPs
Marc Rigter, Paul Duckworth, Bruno Lacerda +1
Planning in Markov decision processes (MDPs) typically optimises the expected cost. However, optimising the expectation does not consider the risk that for any given run of the MDP…
Towards Stochastic Fault-tolerant Control using Precision Learning and Active Inference
Mohamed Baioumy, Corrado Pezzato, Carlos Hernandez Corbato +2
This work presents a fault-tolerant control scheme for sensory faults in robotic manipulators based on active inference. In the majority of existing schemes, a binary decision of w…
On Solving a Stochastic Shortest-Path Markov Decision Process as Probabilistic Inference
Mohamed Baioumy, Bruno Lacerda, Paul Duckworth +1
Previous work on planning as active inference addresses finite horizon problems and solutions valid for online planning. We propose solving the general Stochastic Shortest-Path Mar…