Connection among stochastic Hamilton-Jacobi-Bellman equation, path-integral, and Koopman operator on nonlinear stochastic optimal control

Abstract

The path-integral control, which stems from the stochastic Hamilton-Jacobi-Bellman equation, is one of the methods to control stochastic nonlinear systems. This paper gives a new insight into nonlinear stochastic optimal control problems from the perspective of Koopman operators. When a finite-dimensional dynamical system is nonlinear, the corresponding Koopman operator is linear. Although the Koopman operator is infinite-dimensional, adequate approximation makes it tractable and useful in some discussions and applications. Employing the Koopman operator perspective, it is clarified that only a specific type of observable is enough to be focused on in the control problem. This fact becomes easier to understand via path-integral control. Furthermore, the focus on the specific observable leads to a natural power-series expansion; coupled ordinary differential equations for discrete-state space systems are derived. A demonstration for nonlinear stochastic optimal control shows that the derived equations work well.

arXiv comment12 pages, 4 figures

References in corpus (13)

• A Data-Driven Approximation of the Koopman Operator: Extending Dynamic Mode Decomposition
• Path integrals and symmetry breaking for optimal control theory
• A linear theory for control of non-linear stochastic systems
• Data-driven approximation of the Koopman generator: Model reduction, system identification, and control
• Koopman operator-based model reduction for switched-system control of PDEs
• Shear-Induced Chaos
• Path Integral Control and State Dependent Feedback
• Extended duality relations between birth-death processes and partial differential equations
• Memory effects in the Fermi-Pasta-Ulam Model
• Generalized Langevin equations for systems with local interactions
• Duality in stochastic processes from the viewpoint of basis expansions
• Making Birth-Death Processes from Backward Fokker-Planck Equations for Computing Expectations in Langevin Systems
• Nonlinear Kalman filter based on duality relations between continuous and discrete-state stochastic processes

Cited by in corpus (2)

• Numerical methods to evaluate Koopman matrix from system equations
• Statistics for stochastic differential equations and approximations of resolvent