Zero Collapse: A Failure Mode of Policy Gradient Methods in Discontinuous Reward Environments

Nishant Kumar, Enrique Areyan Viqueira, Amy Greenwald

Policy-gradient learning can deteriorate even after substantial initial improvement. We study this behavior in a repeated first-price auction setting with a sharp threshold between…