Ergodic Theorems for discrete Markov chains

Abstract

Let $X_n$ be a discrete time Markov chain with state space $S$ (countably infinite, in general) and initial probability distribution ${\mu }^{(0)}=(P(X_0=i_1),P(X_0=i_2),\cdots ,)$. What is the probability of choosing in random some $k\in \mathbb{N}$ with $k\le n$ such that $X_k=j$ where $j\in S$? This probability is the average $\frac{1}{n}{\sum }_{k=1}^n{\mu }_j^{(k)}$ where ${\mu }_j^{(k)}=P(X_k=j)$. In this note we will study the limit of this average without assuming that the chain is irreducible, using elementary mathematical tools. Finally, we study the limit of the average $\frac{1}{n}{\sum }_{k=1}^{n}g(X_k)$ where $g$ is a given function for a Markov chain not necessarily irreducible.