Landscape of Policy Optimization for Finite Horizon MDPs with General State and Action

Xin Chen, Yifan Hu, Minda Zhao

Policy gradient methods are widely used in reinforcement learning. Yet, the nonconvexity of policy optimization poses significant challenges in understanding the global convergence…