Hidden States as Value Gradients: The Pontryagin Structure of Recurrent Policies

David Leeftink, Max Hinne, Marcel van Gerven

A key capability of intelligent agents is to act effectively under incomplete state observations. Recurrent policies address this by compressing observation histories into a hidden…