Safety-Constrained Optimal Control and Hamiltonian-Gradient Learning for Systems with Unknown Dynamics

Abstract

We present a framework for safety-constrained optimal control when the plant dynamics are unknown and only an approximate model is available. We apply Pontryagin's Minimum Principle to a model-based problem whose cost penalizes model-plant state mismatch and establish conditions under which the constrained model and plant Hamiltonians have the same minimizer. These conditions show that optimality can be recovered by preserving control-relevant Hamiltonian structure rather than identifying the complete plant dynamics. Motivated by this observation, we develop a fully online method that uses integral trajectory data and policy iteration to learn the dynamics-dependent factors of the plant Hamiltonian gradient directly. Experiments on robotic cruise control demonstrate constrained model-plant control equivalence and online Hamiltonian-gradient reconstruction.

arXiv commentRevised and extended version with additional theoretical and experimental results submitted to ACC 2027