3 citations · 7 across the 12 of their papers we have counts for
16 papers
Can You Trust Your Preference Models? Stackelberg Games for Alignment
Riccardo Formenti, Aditya Agrawal, Vikas Garg
Language models are frequently aligned to human preferences via imperfect evaluators whose errors can be exploited during optimization. We train policies to search for comparisons…
Adaptive Policy Portfolios for Robust Markov Decision Processes
Kasper Engelen, Sebastian Junges, Guillermo A. Pérez +1
Robust Markov decision processes optimize one policy against a set of plausible transition functions. This can be conservative when the unknown dynamics are fixed and become partia…
On the Complexity of Robust Markov Decision Processes and Bisimulation Metrics
Marnix Suilen, Guillermo A. Pérez
Robust Markov decision processes (RMDPs) extend standard Markov decision processes (MDPs) to account for uncertainty in the transition probabilities. RMDPs have an uncertainty set…
Multi-Environment POMDPs: Discrete Model Uncertainty Under Partial Observability
Eline M. Bovy, Caleb Probine, Marnix Suilen +2
Multi-environment POMDPs (ME-POMDPs) extend standard POMDPs with discrete model uncertainty. ME-POMDPs represent a finite set of POMDPs that share the same state, action, and obser…
Data-Efficient Safe Policy Improvement Using Parametric Structure
Kasper Engelen, Guillermo A. Pérez, Marnix Suilen
Safe policy improvement (SPI) is an offline reinforcement learning problem in which a new policy that reliably outperforms the behavior policy with high confidence needs to be comp…
Robust Markov Decision Processes: A Place Where AI and Formal Methods Meet
Marnix Suilen, Thom Badings, Eline M. Bovy +2
Markov decision processes (MDPs) are a standard model for sequential decision-making problems and are widely used across many scientific areas, including formal methods and artific…
Pessimistic Iterative Planning with RNNs for Robust POMDPs
Maris F. L. Galesloot, Marnix Suilen, Thiago D. Simão +4
Robust POMDPs extend classical POMDPs to incorporate model uncertainty using so-called uncertainty sets on the transition and observation functions, effectively defining ranges of…
A PSPACE Algorithm for Almost-Sure Rabin Objectives in Multi-Environment MDPs
Marnix Suilen, Marck van der Vegt, Sebastian Junges
Markov Decision Processes (MDPs) model systems with uncertain transition dynamics. Multiple-environment MDPs (MEMDPs) extend MDPs. They intuitively reflect finite sets of MDPs that…
Imprecise Probabilities Meet Partial Observability: Game Semantics for Robust POMDPs
Eline M. Bovy, Marnix Suilen, Sebastian Junges +1
Partially observable Markov decision processes (POMDPs) rely on the key assumption that probability distributions are precisely known. Robust POMDPs (RPOMDPs) alleviate this concer…
More for Less: Safe Policy Improvement With Stronger Performance Guarantees
Patrick Wienhöft, Marnix Suilen, Thiago D. Simão +3
In an offline reinforcement learning setting, the safe policy improvement (SPI) problem aims to improve the performance of a behavior policy according to which sample data has been…
Decision-Making Under Uncertainty: Beyond Probabilities
Thom Badings, Thiago D. Simão, Marnix Suilen +1
This position paper reflects on the state-of-the-art in decision-making under uncertainty. A classical assumption is that probabilities can sufficiently capture all uncertainty in…
Safe Policy Improvement for POMDPs via Finite-State Controllers
Thiago D. Simão, Marnix Suilen, Nils Jansen
We study safe policy improvement (SPI) for partially observable Markov decision processes (POMDPs). SPI is an offline reinforcement learning (RL) problem that assumes access to (1)…
Robust Anytime Learning of Markov Decision Processes
Marnix Suilen, Thiago D. Simão, David Parker +1
Markov decision processes (MDPs) are formal models commonly used in sequential decision-making. MDPs capture the stochasticity that may arise, for instance, from imprecise actuator…
Balancing Wind and Batteries: Towards Predictive Verification of Smart Grids
Thom S. Badings, Arnd Hartmanns, Nils Jansen +1
We study a smart grid with wind power and battery storage. Traditionally, day-ahead planning aims to balance demand and wind power, yet actual wind conditions often deviate from fo…
Robust Finite-State Controllers for Uncertain POMDPs
Murat Cubuktepe, Nils Jansen, Sebastian Junges +3
Uncertain partially observable Markov decision processes (uPOMDPs) allow the probabilistic transition and observation functions of standard POMDPs to belong to a so-called uncertai…
Robust Policy Synthesis for Uncertain POMDPs via Convex Optimization
Marnix Suilen, Nils Jansen, Murat Cubuktepe +1
We study the problem of policy synthesis for uncertain partially observable Markov decision processes (uPOMDPs). The transition probability function of uPOMDPs is only known to bel…