52 citations · 146 across the 41 of their papers we have counts for
7 papers · 1 filter
Transience in Countable MDPs
Stefan Kiefer, Richard Mayr, Mahsa Shirmohammadi +1
The Transience objective is not to visit any state infinitely often. While this is not possible in finite Markov Decision Process (MDP), it can be satisfied in countably infinite o…
Online Monitoring -Regular Properties in Unknown Markov Chains
Javier Esparza, Stefan Kiefer, Jan Kretinsky +1
We study runtime monitoring of -regular properties. We consider a simple setting in which a run of an unknown finite-state Markov chain is monitored against a fixed…
Equivalence of Hidden Markov Models with Continuous Observations
Oscar Darwin, Stefan Kiefer
We consider Hidden Markov Models that emit sequences of observations that are drawn from continuous distributions. For example, such a model may emit a sequence of numbers, each of…
Comparing Labelled Markov Decision Processes
Stefan Kiefer, Qiyi Tang
A labelled Markov decision process is a labelled Markov chain with nondeterminism, i.e., together with a strategy a labelled MDP induces a labelled Markov chain. The model is relat…
Notes on Equivalence and Minimization of Weighted Automata
Stefan Kiefer
This set of notes re-proves known results on weighted automata (over a field, also known as multiplicity automata). The text offers a unified view on theorems and proofs that have…
Strategy Complexity of Parity Objectives in Countable MDPs
Stefan Kiefer, Richard Mayr, Mahsa Shirmohammadi +1
We study countably infinite MDPs with parity objectives. Unlike in finite MDPs, optimal strategies need not exist, and may require infinite memory if they do. We provide a complete…