Markov Chain Approximation of Sticky Diffusions and Hamilton-Jacobi-Bellman Equations on Networks
arXiv:2607.26709
The paper introduces a discrete Markov‑chain method to approximate diffusions on networks with Kirchhoff and sticky vertex conditions, proves its convergence, and uses it to build a semi‑Lagrangian scheme for Hamilton‑Jacobi‑Bellman equations on networks with convergence guarantees via viscosity solutions.
Abstract
We propose a discrete Markov-chain approximation of diffusion processes on networks with both Kirchhoff and sticky vertex conditions. Stickiness is modeled by a probabilistic residence mechanism at the vertex, while the motion along the edges follows an Euler-Maruyama-type update at the diffusive scale. We prove that the associated time-interpolated chain converges in distribution to the limiting diffusion in the Skorokhod space using the Ethier-Kurtz framework. Based on this construction, we derive a fully discrete semi-Lagrangian scheme for Hamilton-Jacobi-Bellman equations on networks and establish its convergence using viscosity solution techniques.