Hitting Times and Probabilities for Imprecise Markov Chains

Abstract

We consider the problem of characterising expected hitting times and hitting probabilities for imprecise Markov chains. To this end, we consider three distinct ways in which imprecise Markov chains have been defined in the literature: as sets of homogeneous Markov chains, as sets of more general stochastic processes, and as game-theoretic probability models. Our first contribution is that all these different types of imprecise Markov chains have the same lower and upper expected hitting times, and similarly the hitting probabilities are the same for these three types. Moreover, we provide a characterisation of these quantities that directly generalises a similar characterisation for precise, homogeneous Markov chains.

References in corpus (1)

• Continuity Properties of Game-theoretic Upper Expectations

Cited by in corpus (4)

• Global Upper Expectations for Discrete-Time Stochastic Processes: In Practice, They Are All The Same!
• Computing Expected Hitting Times for Imprecise Markov Chains
• Faster Algorithms for Quantitative Analysis of Markov Chains and Markov Decision Processes with Small Treewidth
• Robust Model Checking with Imprecise Markov Reward Models