4 papers · 1 filter
Mirror Descent for Stochastic Control Problems with Measure-valued Controls
Bekzhan Kerimkulov, David Šiška, Łukasz Szpruch +1
This paper studies the convergence of the mirror descent algorithm for finite horizon stochastic control problems with measure-valued control processes. The control objective invol…
A Fisher-Rao gradient flow for entropy-regularised Markov decision processes in Polish spaces
Bekzhan Kerimkulov, James-Michael Leahy, David Siska +2
We study the global convergence of a Fisher-Rao policy gradient flow for infinite-horizon entropy-regularised Markov decision processes with Polish state and action space. The flow…
A modified MSA for stochastic control problems
Bekzhan Kerimkulov, David Šiška, Łukasz Szpruch
The classical Method of Successive Approximations (MSA) is an iterative method for solving stochastic control problems and is derived from Pontryagin's optimality principle. It is…
Exponential Convergence and stability of Howards's Policy Improvement Algorithm for Controlled Diffusions
B. Kerimkulov, D. Šiška, Ł. Szpruch
Optimal control problems are inherently hard to solve as the optimization must be performed simultaneously with updating the underlying system. Starting from an initial guess, Howa…