On the continuity and smoothness of the value function in reinforcement learning and optimal control

Hans Harder, Sebastian Peitz

The value function plays a crucial role as a measure for the cumulative future reward an agent receives in both reinforcement learning and optimal control. It is therefore of inter…