Imprecise Markov chains and their limit behaviour

Abstract

When the initial and transition probabilities of a finite Markov chain in discrete time are not well known, we should perform a sensitivity analysis. This can be done by considering as basic uncertainty models the so-called credal sets that these probabilities are known or believed to belong to, and by allowing the probabilities to vary over such sets. This leads to the definition of an imprecise Markov chain. We show that the time evolution of such a system can be studied very efficiently using so-called lower and upper expectations, which are equivalent mathematical representations of credal sets. We also study how the inferred credal set about the state at time n evolves as n goes to infinity: under quite unrestrictive conditions, it converges to a uniquely invariant credal set, regardless of the credal set given for the initial state. This leads to a non-trivial generalisation of the classical Perron-Frobenius Theorem to imprecise Markov chains.

v1: 28 pages, 8 figures; v2: 31 pages, 9 figures, major revision after review: added, modified, and removed material (no results dropped, results added), moved proofs to an appendix

References in corpus (1)

• n-Monotone exact functionals

Cited by in corpus (11)

• Extended Laplace Principle for Empirical Measures of a Markov Chain
• A classification of invariant distributions and convergence of imprecise Markov chains
• Random walks on graphs with interval weights and precise marginals
• Stability and convergence in discrete convex monotone dynamical systems
• Normal cones corresponding to credal sets of lower probabilities
• Computing bounds for imprecise continuous-time Markov chains using normal cones
• Computing Expected Hitting Times for Imprecise Markov Chains
• Constructing copulas from shock models with imprecise distributions
• Errors bounds for finite approximations of coherent lower previsions on finite probability spaces
• Perturbation bounds and degree of imprecision for uniquely convergent imprecise Markov chains
• Upper Expected Meeting Times for Interdependent Stochastic Agents