A computational approach to extreme values and related hitting probabilities in level-dependent quasi-birth-death processes

Abstract

This paper analyzes the dynamics of a level-dependent quasi-birth-death process $\mathcal{X}=\{(I(t),J(t)):t\ge 0\}$, i.e., a bi-variate Markov chain defined on the countable state space ${\cup }_{i=0}^{\infty }l(i)$ with $l(i)=\{(i,j):j\in \{0,...,M_i\}\}$, for integers $M_i\in {\mathbb{N}}_0$ and $i\in {\mathbb{N}}_0$, which has the special property that its $q$-matrix has a block-tridiagonal form. Under the assumption that the first passage to the subset $l(0)$ occurs in a finite time with certainty, we characterize the probability law of $(\ddot{I}{„}_{\max },I_{\max },J(\ddot{I}{„}_{\max }))$, where $I_{\max }$ is the running maximum level attained by process $\mathcal{X}$ before its first visit to states in $l(0)$, $\ddot{I}{„}_{\max }$ is the first time that the level process $\{I(t):t\ge 0\}$ reaches the running maximum $I_{\max }$, and $J(\ddot{I}{„}_{\max })$ is the phase at time $\ddot{I}{„}_{\max }$. Our methods rely on the use of restricted Laplace-Stieltjes transforms of $\ddot{I}{„}_{\max }$ on the set of sample paths $\{I_{\max }=i,J(\ddot{I}{„}_{\max })=j\}$, and related processes under taboo of certain subsets of states. The utility of the resulting computational algorithms is demonstrated in two epidemic models: the SIS model for horizontally and vertically transmitted diseases; and the SIR model with constant population size.

28 pages, 5 figures