Learn Quasi-stationary Distributions of Finite State Markov Chain
arXiv:2111.11213 · doi:10.3390/e24010133
Abstract
We propose a reinforcement learning (RL) approach to compute the expression of quasi-stationary distribution. Based on the fixed-point formulation of quasi-stationary distribution, we minimize the KL-divergence of two Markovian path distributions induced by the candidate distribution and the true target distribution. To solve this challenging minimization problem by gradient descent, we apply the reinforcement learning technique by introducing the reward and value functions. We derive the corresponding policy gradient theorem and design an actor-critic algorithm to learn the optimal solution and the value function. The numerical examples of finite state Markov chain are tested to demonstrate the new method.
arXiv comment18 pages, 5 figures
References in corpus (11)
- Variational Inference: A Review for Statisticians
- Deep Reinforcement Learning for De-Novo Drug Design
- Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm
- Quasi-stationary distributions and population processes
- Mixed-mode oscillations and interspike interval statistics in the stochastic FitzHugh-Nagumo model
- A reinforcement learning approach to rare trajectory sampling
- Jump Markov models and transition state theory: the Quasi-Stationary Distribution approach
- A stochastic approximation approach to quasi-stationary distributions on finite spaces
- Low temperature asymptotics for Quasi-Stationary Distributions in a bounded domain
- Quasi-Stationary Distributions for Models of Heterogeneous Catalysis
- An approximation scheme for quasi-stationary distributions of killed diffusions