A Multistep Lyapunov Approach for Finite-Time Analysis of Biased Stochastic Approximation

Gang Wang, Bingcong Li, Georgios B. Giannakis

Motivated by the widespread use of temporal-difference (TD-) and Q-learning algorithms in reinforcement learning, this paper studies a class of biased stochastic approximation (SA)…