Global Convergence of the ODE Limit for Online Actor-Critic Algorithms in Reinforcement Learning

Ziheng Wang, Justin Sirignano

Actor-critic algorithms are widely used in reinforcement learning, but are challenging to mathematically analyse due to the online arrival of non-i.i.d. data samples. The distribut…