Linearly Solvable Stochastic Control Lyapunov Functions
arXiv:1410.0405 · doi:10.1137/16M105767X
Abstract
This paper presents a new method for synthesizing stochastic control Lyapunov functions for a class of nonlinear stochastic control systems. The technique relies on a transformation of the classical nonlinear Hamilton-Jacobi-Bellman partial differential equation to a linear partial differential equation for a class of problems with a particular constraint on the stochastic forcing. This linear partial differential equation can then be relaxed to a linear differential inclusion, allowing for relaxed solutions to be generated using sum of squares programming. The resulting relaxed solutions are in fact viscosity super/subsolutions, and by the maximum principle are pointwise upper and lower bounds to the underlying value function, even for coarse polynomial approximations. Furthermore, the pointwise upper bound is shown to be a stochastic control Lyapunov function, yielding a method for generating nonlinear controllers with pointwise bounded distance from the optimal cost when using the optimal controller. These approximate solutions may be computed with non-increasing error via a hierarchy of semidefinite optimization problems. Finally, this paper develops a-priori bounds on trajectory suboptimality when using these approximate value functions, as well as demonstrates that these methods, and bounds, can be applied to a more general class of nonlinear systems not obeying the constraint on stochastic forcing. Simulated examples illustrate the methodology.
arXiv commentPublished in SIAM Journal of Control and Optimization
References in corpus (5)
- Path integrals and symmetry breaking for optimal control theory
- A linear theory for control of non-linear stochastic systems
- Suboptimal Stabilizing Controllers for Linearly Solvable System
- Relative Entropy Relaxations for Signomial Optimization
- Semidefinite Relaxations for Stochastic Optimal Control Policies