Computing the Bias of Constant-step Stochastic Approximation with Markovian Noise

Sebastian Allmeier, Nicolas Gast

We study stochastic approximation algorithms with Markovian noise and constant step-size $\hat{I}\pm$. We develop a method based on infinitesimal generator comparisons to study the bias of…