Deterministic particle flows for constraining SDEs
arXiv:2110.13020
Abstract
Devising optimal interventions for diffusive systems often requires the solution of the Hamilton-Jacobi-Bellman (HJB) equation, a nonlinear backward partial differential equation (PDE), that is, in general, nontrivial to solve. Existing control methods either tackle the HJB directly with grid-based PDE solvers, or resort to iterative stochastic path sampling to obtain the necessary controls. Here, we present a framework that interpolates between these two approaches. By reformulating the optimal interventions in terms of logarithmic gradients ( scores ) of two forward probability flows, and by employing deterministic particle methods for solving Fokker-Planck equations, we introduce a novel fully deterministic framework that computes the required optimal interventions in one shot.
arXiv comment4+3 pages, 2 figures -- workshop paper -- Machine Learning and the Physical Sciences, Workshop at the 35th Conference on Neural Information Processing Systems (NeurIPS 2021) The related journal paper is published here: https://link.aps.org/doi/10.1103/PhysRevResearch.4.043035
References in corpus (10)
- Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review
- A linear theory for control of non-linear stochastic systems
- Optimal control as a graphical model inference problem
- Adaptive importance sampling for control and inference
- Control of stochastic and induced switching in biophysical networks
- Learning Optimal Control via Forward and Backward Stochastic Differential Equations
- Interacting particle solutions of Fokker-Planck equations through gradient-log-density estimation
- Solving non-linear Kolmogorov equations in large dimensions by using deep learning: a numerical comparison of discretization schemes
- An Optimal Control Derivation of Nonlinear Smoothing Equations
- Schrödinger Bridge Samplers