Convergence of Two-Timescale Markovian Stochastic Approximations with Applications in Reinforcement Learning

Vagul Mahadevan, Claire Chen, Shuze Daniel Liu +1

This work studies the convergence of two-timescale stochastic approximations (SA), a class of iterative algorithms that update two sets of parameters in fast and slow timescales re…