59 citations · 291 across the 45 of their papers we have counts for
17 papers · 1 filter
Property-driven Causal Abstractions for Markov Decision Processes
Jule Schmidt, Maximilian Weininger, Clemens Dubslaff +2
Markov Decision Processes (MDPs) are widely used as decision-making models, commonly specified over factored state spaces through state variables and their valuations. The exponent…
Act-Then-Measure: Reinforcement Learning for Partially Observable Environments with Active Measuring
Merlijn Krale, Thiago D. Simão, Nils Jansen
We study Markov decision processes (MDPs), where agents have direct control over when and how they gather information, as formalized by action-contingent noiselessly observable MDP…
Decision-Making Under Uncertainty: Beyond Probabilities
Thom Badings, Thiago D. Simão, Marnix Suilen +1
This position paper reflects on the state-of-the-art in decision-making under uncertainty. A classical assumption is that probabilities can sufficiently capture all uncertainty in…
Safe Policy Improvement for POMDPs via Finite-State Controllers
Thiago D. Simão, Marnix Suilen, Nils Jansen
We study safe policy improvement (SPI) for partially observable Markov decision processes (POMDPs). SPI is an offline reinforcement learning (RL) problem that assumes access to (1)…
Robust Anytime Learning of Markov Decision Processes
Marnix Suilen, Thiago D. Simão, David Parker +1
Markov decision processes (MDPs) are formal models commonly used in sequential decision-making. MDPs capture the stochasticity that may arise, for instance, from imprecise actuator…
Safe Reinforcement Learning via Shielding under Partial Observability
Steven Carr, Nils Jansen, Sebastian Junges +1
Safe exploration is a common problem in reinforcement learning (RL) that aims to prevent agents from making disastrous decisions while exploring their environment. A family of appr…
Correct-by-construction reach-avoid control of partially observable linear stochastic systems
Thom Badings, Hasan A. Poonawala, Marielle Stoelinga +1
We study feedback controller synthesis for reach-avoid control of discrete-time, linear time-invariant (LTI) systems with Gaussian process and measurement noise. The problem is to…
Robust Finite-State Controllers for Uncertain POMDPs
Murat Cubuktepe, Nils Jansen, Sebastian Junges +3
Uncertain partially observable Markov decision processes (uPOMDPs) allow the probabilistic transition and observation functions of standard POMDPs to belong to a so-called uncertai…
Strengthening Deterministic Policies for POMDPs
Leonore Winterer, Ralf Wimmer, Nils Jansen +1
The synthesis problem for partially observable Markov decision processes (POMDPs) is to compute a policy that satisfies a given specification. Such policies have to take the full e…
Enforcing Almost-Sure Reachability in POMDPs
Sebastian Junges, Nils Jansen, Sanjit A. Seshia
Partially-Observable Markov Decision Processes (POMDPs) are a well-known stochastic model for sequential decision making under limited information. We consider the EXPTIME-hard pro…
Neural Simplex Architecture
Dung T. Phan, Radu Grosu, Nils Jansen +3
We present the Neural Simplex Architecture (NSA), a new approach to runtime assurance that provides safety guarantees for neural controllers (obtained e.g. using reinforcement lear…
Counterexample-Guided Strategy Improvement for POMDPs Using Recurrent Neural Networks
Steven Carr, Nils Jansen, Ralf Wimmer +3
We study strategy synthesis for partially observable Markov decision processes (POMDPs). The particular problem is to determine strategies that provably adhere to (probabilistic) t…
The Partially Observable Games We Play for Cyber Deception
Mohamadreza Ahmadi, Murat Cubuktepe, Nils Jansen +3
Progressively intricate cyber infiltration mechanisms have made conventional means of defense, such as firewalls and malware detectors, incompetent. These sophisticated infiltratio…
Safe Reinforcement Learning via Probabilistic Shields
Nils Jansen, Bettina Könighofer, Sebastian Junges +2
This paper targets the efficient construction of a safety shield for decision making in scenarios that incorporate uncertainty. Markov decision processes (MDPs) are prominent model…
Synthesis in pMDPs: A Tale of 1001 Parameters
Murat Cubuktepe, Nils Jansen, Sebastian Junges +2
This paper considers parametric Markov decision processes (pMDPs) whose transitions are equipped with affine functions over a finite set of parameters. The synthesis problem is to…
Human-in-the-Loop Synthesis for Partially Observable Markov Decision Processes
Steven Carr, Nils Jansen, Ralf Wimmer +2
We study planning problems where autonomous agents operate inside environments that are subject to uncertainties and not fully observable. Partially observable Markov decision proc…
Probabilistic Model Checking for Complex Cognitive Tasks -- A case study in human-robot interaction
Sebastian Junges, Nils Jansen, Joost-Pieter Katoen +1
This paper proposes to use probabilistic model checking to synthesize optimal robot policies in multi-tasking autonomous systems that are subject to human-robot interaction. Given…