A Note on Exact State Visit Probabilities in Two-State Markov Chains

Mohammad Taha Shah

In this note we derive the exact probability that a specific state in a two-state Markov chain is visited exactly $k$ times after $N$ transitions. We provide a closed-form solution…