Occupation laws for some time-nonhomogeneous Markov chains

Abstract

We consider finite-state time-nonhomogeneous Markov chains where the probability of moving from state $i$ to state $j\ne i$ at time $n$ is $G(i,j)/n^{\zeta }$ for a ``generator'' matrix $G$ and strength parameter $\zeta >0$. In these chains, as time grows, the positions are less and less likely to change, and so form simple models of age-dependent time-reinforcing behaviors. These chains, however, exhibit some different, perhaps unexpected, asymptotic occupation laws depending on parameters. Although on the one hand it is shown that the asymptotic position converges to a point-mixture for all $\zeta >0$, on the other hand, the average position, when variously $0<\zeta <1$, $\zeta >1$ or $\zeta =1$, is shown to converges to a constant, a point-mixture, or a distribution ${\mu }_G$ with no atoms and full support on a certain simplex respectively. The last type of limit can be seen as a sort of ``spreading'' between the cases $0<\zeta <1$ and $\zeta >1$. In particular, when $G$ is appropriately chosen, ${\mu }_G$ is a Dirichlet distribution with certain parameters, reminiscent of results in Polya urns.

24 pages, 2 figures