The Derivation of Markov Chain Properties using Generalized Matrix Inverses

Abstract

The analysis of many problems of interest associated with Markov chains, e.g. stationary distributions, moments of first passage time distributions and moments of occupation time random variables, involves the solution of a system of linear equations involving I - P, where P is the transition matrix of a finite, irreducible, discrete time Markov chain. Generalized inverses play an important role in the solution of such singular sets of equations. In this presentation we survey the application of generalized matrix inverses to the aforementioned problems.

arXiv comment31 pages