Why does the two-timescale Q-learning converge to different mean field solutions? A unified convergence analysis

Jing An, Jianfeng Lu, Yue Wu +1

We revisit the unified two-timescale Q-learning algorithm as initially introduced by Angiuli et al. \cite{angiuli2022unified}. This algorithm demonstrates efficacy in solving mean…