The Cesà ro Value Iteration

Abstract

In this paper, we consider undiscouted infinite-horizon optimal control for deterministic systems with an uncountable state and input space. We specifically address the case when the classic value iteration does not converge. For such systems, we use the Ces`aro mean to define the infinite-horizon optimal control problem and the corresponding infinite-horizon value function. Moreover, for this value function, we introduce the Cesà ro value iteration and prove its convergence for the special case of systems with periodic optimal operating behavior. For this instance, we also show that the Cesà ro value function recovers the undiscounted infinite-horizon optimal cost, if the latter is well-defined.

Extended version of "The Cesà ro Value Iteration", IEEE Control Systems Letters, vol.9, pp.1934-1939, 2025 (DOI: 10.1109/LCSYS.2025.3584792)