Pontryagin's Minimum Principle and Forward-Backward Sweep Method for the System of HJB-FP Equations in Memory-Limited Partially Observable Stochastic Control
arXiv:2210.13040 · doi:10.3390/e25020208
Abstract
Memory-limited partially observable stochastic control (ML-POSC) is the stochastic optimal control problem under incomplete information and memory limitation. In order to obtain the optimal control function of ML-POSC, a system of the forward Fokker-Planck (FP) equation and the backward Hamilton-Jacobi-Bellman (HJB) equation needs to be solved. In this work, we firstly show that the system of HJB-FP equations can be interpreted via the Pontryagin's minimum principle on the probability density function space. Based on this interpretation, we then propose the forward-backward sweep method (FBSM) to ML-POSC, which has been used in the Pontryagin's minimum principle. FBSM is an algorithm to compute the forward FP equation and the backward HJB equation alternately. Although the convergence of FBSM is generally not guaranteed, it is guaranteed in ML-POSC because the coupling of HJB-FP equations is limited to the optimal control function in ML-POSC.
References in corpus (4)
Cited by in corpus (7)
- Memory-Limited Partially Observable Stochastic Control and its Mean-Field Control Approach
- Rational social distancing policy during epidemics with limited healthcare capacity
- Mean-Field Control Approach to Decentralized Stochastic Control with Finite-Dimensional Memories
- Rational social distancing in epidemics with uncertain vaccination timing
- Theory for Optimal Estimation and Control under Resource Limitations and Its Applications to Biological Information Processing and Decision-Making
- Resource Limitations induce Phase Transitions in Biological Information Processing
- Theoretical Analysis of Resource-Induced Phase Transitions in Estimation Strategies