Stationary distributions of continuous-time Markov chains: a review of theory and truncation-based approximations
arXiv:1909.05794
Abstract
Computing the stationary distributions of a continuous-time Markov chain (CTMC) involves solving a set of linear equations. In most cases of interest, the number of equations is infinite or too large, and the equations cannot be solved analytically or numerically. Several approximation schemes overcome this issue by truncating the state space to a manageable size. In this review, we first give a comprehensive theoretical account of the stationary distributions and their relation to the long-term behaviour of CTMCs that is readily accessible to non-experts and free of irreducibility assumptions made in standard texts. We then review truncation-based approximation schemes for CTMCs with infinite state spaces paying particular attention to the schemes' convergence and the errors they introduce, and we illustrate their performance with an example of a stochastic reaction network of relevance in biology and chemistry. We conclude by discussing computational trade-offs associated with error control and several open questions.
References in corpus (14)
- Analytical distributions for stochastic gene expression
- Predator-prey cycles from resonant amplification of demographic stochasticity
- Genetic Toggle Switch Without Cooperative Binding
- Exact distributions for stochastic gene expression models with bursting and feedback
- A scalable computational framework for establishing long-term behavior of stochastic reaction networks
- Stochastic models of gene transcription with upstream drives: exact solution and sample path characterization
- Stability and multi-attractor dynamics of a toggle switch based on a two-stage model of stochastic gene expression
- Perfect Sampling of the Master Equation for Gene Regulatory Networks
- Error bounds for last-column-block-augmented truncations of block-structured Markov chains
- Approximating Markov chains and V-geometric ergodicity via weak perturbation theory
- Continuous-time block-monotone Markov chains and their block-augmented truncations
- Bounding stationary averages of polynomial diffusions via semidefinite programming
- Limit formulas for the normalized fundamental matrix of the northwest-corner truncation of Markov chains: Matrix-infinite-product-form solutions of block-Hessenberg Markov chains
- Markov chains revisited