333 citations · 684 across the 37 of their papers we have counts for
40 papers
Optimizing Minimax Regret in Uncertain MDPs with Small Sets of Policies
Sterre Lutz, Daniël Vos, Matthijs T. J. Spaan +1
Sequential decision-making in real-world applications often involves uncertainty about the environment's model. Uncertain Markov decision processes (UMDPs) represent the possible e…
Generalization in offline RL: The structure is more important than the amount of pessimism
Max Weltevrede, Matthijs T. J. Spaan, Wendelin Böhmer
While pessimism counteracts overestimation bias in offline reinforcement learning (RL), being overly conservative has been associated with hindering certain forms of generalization…
COP-Q: Safety-First Reinforcement Learning for Robot Control via Cholesky-Ordered Projection
Guopeng Li, Moritz A. Zanger, Matthijs T. J. Spaan +1
Safe robot control requires maximizing return while satisfying safety constraints. In off-policy safe reinforcement learning, reward and safety Q-values are commonly learned by sep…
AlphaExploitem: Going Beyond the Nash Equilibrium in Poker by Learning to Exploit Suboptimal Play
Vlad Murgoci, Matthijs Spaan, Yaniv Oren
Poker is an imperfect information game that has served as a long-standing benchmark for decision-making under uncertainty. To maximize utility beyond the Nash equilibrium, an agent…
Particle Monte Carlo Tree Search
Yaniv Oren, Viliam Vadocz, Joery A. de Vries +3
Monte Carlo Tree Search (MCTS) is a widely used approach for policy improvement and action selection in Reinforcement Learning. Due to its sequential and deterministic nature, prin…
Off-Policy Safe Reinforcement Learning with Constrained Optimistic Exploration
Guopeng Li, Matthijs T. J. Spaan, Julian F. P. Kooij
When safety is formulated as a limit of cumulative cost, safe reinforcement learning (RL) aims to learn policies that maximize return subject to the cost constraint in data collect…
On the Equivalence of Random Network Distillation, Deep Ensembles, and Bayesian Inference
Moritz A. Zanger, Yijun Wu, Pascal R. Van der Vaart +2
Uncertainty quantification is central to safe and efficient deployments of deep learning models, yet many computationally practical methods lack lacking rigorous theoretical motiva…
Sparse Masked Attention Policies for Reliable Generalization
Caroline Horsch, Laurens Engwegen, Max Weltevrede +2
In reinforcement learning, abstraction methods that remove unnecessary information from the observation are commonly used to learn policies which generalize better to unseen tasks.…
VariBASed: Variational Bayes-Adaptive Sequential Monte-Carlo Planning for Deep Reinforcement Learning
Joery A. de Vries, Jinke He, Yaniv Oren +3
Optimally trading-off exploration and exploitation is the holy grail of reinforcement learning as it promises maximal data-efficiency for solving any task. Bayes-optimal agents ach…
Twice Sequential Monte Carlo for Tree Search
Yaniv Oren, Joery A. de Vries, Pascal R. van der Vaart +2
Model-based reinforcement learning (RL) methods that leverage search are responsible for many milestone breakthroughs in RL. Sequential Monte Carlo (SMC) recently emerged as an alt…
Priors Matter: Addressing Misspecification in Bayesian Deep Q-Learning
Pascal R. van der Vaart, Neil Yorke-Smith, Matthijs T. J. Spaan
Uncertainty quantification in reinforcement learning can greatly improve exploration and robustness. Approximate Bayesian approaches have recently been popularized to quantify unce…
RecBayes: Recurrent Bayesian Ad Hoc Teamwork in Large Partially Observable Domains
João G. Ribeiro, Yaniv Oren, Alberto Sardinha +2
This paper proposes RecBayes, a novel approach for ad hoc teamwork under partial observability, a setting where agents are deployed on-the-fly to environments where pre-existing te…
Universal Value-Function Uncertainties
Moritz A. Zanger, Max Weltevrede, Yaniv Oren +4
Estimating epistemic uncertainty in value functions is a crucial challenge for many aspects of reinforcement learning (RL), including efficient exploration, safe decision-making, a…
Bayesian Meta-Reinforcement Learning with Laplace Variational Recurrent Networks
Joery A. de Vries, Jinke He, Mathijs M. de Weerdt +1
Meta-reinforcement learning trains a single reinforcement learning agent on a distribution of tasks to quickly generalize to new tasks outside of the training set at test time. Fro…
VeRecycle: Reclaiming Guarantees from Probabilistic Certificates for Stochastic Dynamical Systems after Change
Sterre Lutz, Matthijs T. J. Spaan, Anna Lukina
Autonomous systems operating in the real world encounter a range of uncertainties. Probabilistic neural Lyapunov certification is a powerful approach to proving safety of nonlinear…
How Ensembles of Distilled Policies Improve Generalisation in Reinforcement Learning
Max Weltevrede, Moritz A. Zanger, Matthijs T. J. Spaan +1
In the zero-shot policy transfer setting in reinforcement learning, the goal is to train an agent on a fixed set of training environments so that it can generalise to similar, but…
Trust-Region Twisted Policy Improvement
Joery A. de Vries, Jinke He, Yaniv Oren +1
Monte-Carlo tree search (MCTS) has driven many recent breakthroughs in deep reinforcement learning (RL). However, scaling MCTS to parallel compute has proven challenging in practic…
Contextual Similarity Distillation: Ensemble Uncertainties with a Single Model
Moritz A. Zanger, Pascal R. Van der Vaart, Wendelin Böhmer +1
Uncertainty quantification is a critical aspect of reinforcement learning and deep learning, with numerous applications ranging from efficient exploration and stable offline reinfo…
Positive Experience Reflection for Agents in Interactive Text Environments
Philip Lippmann, Matthijs T. J. Spaan, Jie Yang
Intelligent agents designed for interactive environments face significant challenges in text-based games, a domain that demands complex reasoning and adaptability. While agents bas…
Training on Irrelevant States Implies Data Augmentation: Generalization in Contextual MDPs
Max Weltevrede, Caroline Horsch, Matthijs T. J. Spaan +1
In the zero-shot policy transfer (ZSPT) setting for contextual Markov decision processes (CMDP), agents train on a fixed, finite set of contexts and must generalize to new ones. Re…
Pessimistic Iterative Planning with RNNs for Robust POMDPs
Maris F. L. Galesloot, Marnix Suilen, Thiago D. Simão +4
Robust POMDPs extend classical POMDPs to incorporate model uncertainty using so-called uncertainty sets on the transition and observation functions, effectively defining ranges of…
Explore-Go: Leveraging Exploration for Generalisation in Deep Reinforcement Learning
Max Weltevrede, Felix Kaubek, Matthijs T. J. Spaan +1
One of the remaining challenges in reinforcement learning is to develop agents that can generalise to novel scenarios they might encounter once deployed. This challenge is often fr…
Value Improved Actor Critic Algorithms
Yaniv Oren, Moritz A. Zanger, Pascal R. van der Vaart +3
To learn approximately optimal acting policies for decision problems, modern Actor Critic algorithms rely on deep Neural Networks (DNNs) to parameterize the acting policy and greed…
When Do Off-Policy and On-Policy Policy Gradient Methods Align?
Davide Mambelli, Stephan Bongers, Onno Zoeter +2
Policy gradient methods are widely adopted reinforcement learning algorithms for tasks with continuous action spaces. These methods succeeded in many application domains, however,…