Neural Temporal-Difference and Q-Learning Provably Converge to Global Optima

Qi Cai, Zhuoran Yang, Jason D. Lee +1

Temporal-difference learning (TD), coupled with neural networks, is among the most fundamental building blocks of deep reinforcement learning. However, due to the nonlinearity in v…