Stochastic Optimal Control as Non-equilibrium Statistical Mechanics: Calculus of Variations over Density and Current

Abstract

In Stochastic Optimal Control (SOC) one minimizes the average cost-to-go, that consists of the cost-of-control (amount of efforts), cost-of-space (where one wants the system to be) and the target cost (where one wants the system to arrive), for a system participating in forced and controlled Langevin dynamics. We extend the SOC problem by introducing an additional cost-of-dynamics, characterized by a vector potential. We propose derivation of the generalized gauge-invariant Hamilton-Jacobi-Bellman equation as a variation over density and current, suggest hydrodynamic interpretation and discuss examples, e.g., ergodic control of a particle-within-a-circle, illustrating non-equilibrium space-time complexity.

4 pages, 1 figure

References in corpus (5)

• Path-integral analysis of fluctuation theorems for general Langevin processes
• Refined Second Law of Thermodynamics for fast random processes
• Statistics of Entropy Production in Linearized Stochastic System
• Explicit solution of relative entropy weighted control
• A Unifying Framework for Linearly Solvable Control

Cited by in corpus (14)

• The geometry of thermodynamic control
• Variational and optimal control representations of conditioned and driven processes
• Population dynamics method with a multi-canonical feedback control
• Effective interactions and large deviations in stochastic processes
• Near-optimal protocols in complex nonequilibrium transformations
• Adaptive sampling of large deviations
• Canonical structure and orthogonality of forces and currents in irreversible Markov chains
• Reinforcement learning of rare diffusive dynamics
• Dynamical large deviations of linear diffusions
• Limited-control optimal protocols arbitrarily far from equilibrium
• Role of current fluctuations in nonreversible samplers
• Predictive Maxwell's Demons
• Stochastic Dynamics of Extended Objects in Driven Systems: I. Higher-Dimensional Currents in the Continuous Setting
• Stochastic Dynamics of Extended Objects in Driven Systems II: Current Quantization in the Low-Temperature Limit