59 citations · 191 across the 36 of their papers we have counts for
48 papers
Learning Formal Specifications from Membership and Preference Queries
Ameesh Shah, Marcell Vazquez-Chanlatte, Sebastian Junges +1
Active learning is a well-studied approach to learning formal specifications, such as automata. In this work, we extend active specification learning by proposing a novel framework…
Search and Explore: Symbiotic Policy Synthesis in POMDPs
Roman Andriushchenko, Alexander Bork, Milan Češka +3
This paper marries two state-of-the-art controller synthesis methods for partially observable Markov decision processes (POMDPs), a prominent model in sequential decision making un…
Efficient Sensitivity Analysis for Parametric Robust Markov Chains
Thom Badings, Sebastian Junges, Ahmadreza Marandi +2
We provide a novel method for sensitivity analysis of parametric robust Markov chains. These models incorporate parameters and sets of probability distributions to alleviate the of…
A Practitioner's Guide to MDP Model Checking Algorithms
Arnd Hartmanns, Sebastian Junges, Tim Quatmann +1
Model checking undiscounted reachability and expected-reward properties on Markov decision processes (MDPs) is key for the verification of systems that act under uncertainty. Popul…
Robust Almost-Sure Reachability in Multi-Environment MDPs
Marck van der Vegt, Nils Jansen, Sebastian Junges
Multiple-environment MDPs (MEMDPs) capture finite sets of MDPs that share the states but differ in the transition dynamics. These models form a proper subclass of partially observa…
COOL-MC: A Comprehensive Tool for Reinforcement Learning and Model Checking
Dennis Gross, Nils Jansen, Sebastian Junges +1
This paper presents COOL-MC, a tool that integrates state-of-the-art reinforcement learning (RL) and model checking. Specifically, the tool builds upon the OpenAI gym and the proba…
Parameter Synthesis in Markov Models: A Gentle Survey
Nils Jansen, Sebastian Junges, Joost-Pieter Katoen
This paper surveys the analysis of parametric Markov models whose transitions are labelled with functions over a finite set of parameters. These models are symbolic representations…
Sampling-Based Verification of CTMCs with Uncertain Rates
Thom S. Badings, Nils Jansen, Sebastian Junges +2
We employ uncertain parametric CTMCs with parametric transition rates and a prior on the parameter values. The prior encodes uncertainty about the actual transition rates, while th…
Abstraction-Refinement for Hierarchical Probabilistic Models
Sebastian Junges, Matthijs T. J. Spaan
Markov decision processes are a ubiquitous formalism for modelling systems with non-deterministic and probabilistic behavior. Verification of these models is subject to the famous…
Probabilistic Program Verification via Inductive Synthesis of Inductive Invariants
Kevin Batz, Mingshuai Chen, Sebastian Junges +3
Essential tasks for the verification of probabilistic programs include bounding expected outcomes and proving termination in finite expected runtime. We contribute a simple yet eff…
Safe Reinforcement Learning via Shielding under Partial Observability
Steven Carr, Nils Jansen, Sebastian Junges +1
Safe exploration is a common problem in reinforcement learning (RL) that aims to prevent agents from making disastrous decisions while exploring their environment. A family of appr…
Inductive Synthesis of Finite-State Controllers for POMDPs
Roman Andriushchenko, Milan Ceska, Sebastian Junges +1
We present a novel learning framework to obtain finite-state controllers (FSCs) for partially observable Markov decision processes and illustrate its applicability for indefinite-h…
Scenario-Based Verification of Uncertain Parametric MDPs
Thom Badings, Murat Cubuktepe, Nils Jansen +3
We consider parametric Markov decision processes (pMDPs) that are augmented with unknown probability distributions over parameter values. The problem is to compute the probability…
A generic physics-informed neural network-based framework for reliability assessment of multi-state systems
Taotao Zhou, Xiaoge Zhang, Enrique Lopez Droguett +1
In this paper, we leverage the recent advances in physics-informed neural network (PINN) and develop a generic PINN-based framework to assess the reliability of multi-state systems…
Gradient-Descent for Randomized Controllers under Partial Observability
Linus Heck, Jip Spel, Sebastian Junges +2
Randomization is a powerful technique to create robust controllers, in particular in partially observable settings. The degrees of randomization have a significant impact on the sy…
Convex Optimization for Parameter Synthesis in MDPs
Murat Cubuktepe, Nils Jansen, Sebastian Junges +2
Probabilistic model checking aims to prove whether a Markov decision process (MDP) satisfies a temporal logic specification. The underlying methods rely on an often unrealistic ass…
Model Checking Finite-Horizon Markov Chains with Probabilistic Inference
Steven Holtzen, Sebastian Junges, Marcell Vazquez-Chanlatte +3
We revisit the symbolic verification of Markov chains with respect to finite horizon reachability properties. The prevalent approach iteratively computes step-bounded state reachab…
Model Repair Revamped: On the Automated Synthesis of Markov Chains
Milan Ceska, Christian Dehnert, Nils Jansen +2
This paper outlines two approaches|based on counterexample-guided abstraction refinement (CEGAR) and counterexample-guided inductive synthesis (CEGIS), respectively to the automate…
Runtime Monitoring for Markov Decision Processes
Sebastian Junges, Hazem Torfah, Sanjit A. Seshia
We investigate the problem of monitoring partially observable systems with nondeterministic and probabilistic dynamics. In such systems, every state may be associated with a risk,…
Entropy-Guided Control Improvisation
Marcell Vazquez-Chanlatte, Sebastian Junges, Daniel J. Fremont +1
High level declarative constraints provide a powerful (and popular) way to define and construct control policies; however, most synthesis algorithms do not support specifying the d…
Inductive Synthesis for Probabilistic Programs Reaches New Horizons
Roman Andriushchenko, Milan Ceska, Sebastian Junges +1
This paper presents a novel method for the automated synthesis of probabilistic programs. The starting point is a program sketch representing a finite family of finite-state Markov…
The Complexity of Reachability in Parametric Markov Decision Processes
Sebastian Junges, Joost-Pieter Katoen, Guillermo A. Pérez +1
This article presents the complexity of reachability decision problems for parametric Markov decision processes (pMDPs), an extension to Markov decision processes (MDPs) where tran…
Robust Finite-State Controllers for Uncertain POMDPs
Murat Cubuktepe, Nils Jansen, Sebastian Junges +3
Uncertain partially observable Markov decision processes (uPOMDPs) allow the probabilistic transition and observation functions of standard POMDPs to belong to a so-called uncertai…
Enforcing Almost-Sure Reachability in POMDPs
Sebastian Junges, Nils Jansen, Sanjit A. Seshia
Partially-Observable Markov Decision Processes (POMDPs) are a well-known stochastic model for sequential decision making under limited information. We consider the EXPTIME-hard pro…