Convergence Analysis of an Upwind Finite-Difference Scheme for Hamilton-Jacobi-Bellman Equations

Abstract

We study the convergence of an upwind finite-difference scheme for Hamilton-Jacobi-Bellman equations on a one-dimensional periodic domain, considering both the value function and the feedback control selected by the discrete pointwise minimization. For classical value functions whose time and spatial derivatives are Hölder continuous in their respective variables, we bound the error at the grid points by a constant times the sum of the time step and spatial mesh width each raised to their common Hölder exponent. This yields first-order convergence when these derivatives are Lipschitz continuous and the meshes are refined proportionally. Under additional spatial regularity of the problem data, we also prove convergence of both one-sided difference quotients of the discrete value function to the spatial derivative of the value function: the integral over space and time of the absolute error tends to zero for each difference quotient. We further establish convergence of the discrete controls to the continuous feedback control when the continuous pointwise minimizer is unique, regardless of how the discrete minimizers are selected. These theoretical results are supported by numerical experiments that illustrate the accuracy of the value-function approximation and the behavior of the computed feedback controls.

arXiv comment41 pages, 8 figures