A linear theory for control of non-linear stochastic systems

Abstract

We address the role of noise and the issue of efficient computation in stochastic optimal control problems. We consider a class of non-linear control problems that can be formulated as a path integral and where the noise plays the role of temperature. The path integral displays symmetry breaking and there exist a critical noise value that separates regimes where optimal control yields qualitatively different solutions. The path integral can be computed efficiently by Monte Carlo integration or by Laplace approximation, and can therefore be used to solve high dimensional stochastic control problems.

5 pages, 3 figures. Accepted to PRL

Cited by in corpus (76)

• Optimal control as a graphical model inference problem
• Thermodynamics as a theory of decision-making with information processing costs
• Variational and optimal control representations of conditioned and driven processes
• Learning universal computations with spikes
• Adaptive importance sampling for control and inference
• Learning Optimal Control via Forward and Backward Stochastic Differential Equations
• Model Predictive Path Integral Control using Covariance Variable Importance Sampling
• Path Integral Control and State Dependent Feedback
• Near-optimal protocols in complex nonequilibrium transformations
• Tube Stochastic Optimal Control for Nonlinear Constrained Trajectory Optimization Problems
• Approximate Bayes learning of stochastic differential equations
• Path Integral Networks: End-to-End Differentiable Optimal Control
• Explore or exploit? A generic model and an exactly solvable case
• Explicit solution of relative entropy weighted control
• Information-Theoretic Bounded Rationality
• Broken symmetry in a two-qubit quantum control landscape
• A Unifying Framework for Linearly Solvable Control
• Particle Smoothing for Hidden Diffusion Processes: Adaptive Path Integral Smoother
• Noise Response Data Reveal Novel Controllability Gramian for Nonlinear Network Dynamics
• Generalization of Uncertainty Relation for Quantum and Stochastic Systems
• Acceleration of Gradient-based Path Integral Method for Efficient Optimal and Inverse Optimal Control
• Path Integral Control by Reproducing Kernel Hilbert Space Embedding
• Stein Variational Model Predictive Control
• Generalized Thompson Sampling for Sequential Decision-Making and Causal Inference
• Effects of water currents on fish migration through a Feynman-type path integral approach under $\sqrt{8/3}$ Liouville-like quantum gravity surfaces
• Sample Efficient Path Integral Control under Uncertainty
• Probabilistic design of optimal sequential decision-making algorithms in learning and control
• Schrödinger Approach to Optimal Control of Large-Size Populations
• Control Barrier Functions for Abstraction-Free Control Synthesis under Temporal Logic Constraints
• Stochastic Variational Method as Quantization Scheme I: Field Quantization of Complex Klein-Gordan Equation
• Uncertainty Relations in Hydrodynamics
• Linearly Solvable Stochastic Control Lyapunov Functions
• Pontryagin's Minimum Principle and Forward-Backward Sweep Method for the System of HJB-FP Equations in Memory-Limited Partially Observable Stochastic Control
• Stochastic Optimal Control in Continuous Space-Time Multi-Agent Systems
• Optimal evolutionary control for artificial selection on molecular phenotypes
• Free Energy and the Generalized Optimality Equations for Sequential Decision Making
• Value Iteration in Continuous Actions, States and Time
• On Entropy Regularized Path Integral Control for Trajectory Optimization
• Distributed Algorithms for Linearly-Solvable Optimal Control in Networked Multi-Agent Systems
• Deep 2FBSDEs For Systems With Control Multiplicative Noise
• Disordered high-dimensional optimal control
• On Kalman-Bucy filters, linear quadratic control and active inference
• Hamilton-Jacobi-Bellman Equations for Maximum Entropy Optimal Control
• Optimal trajectory tracking
• Model-based Path Integral Stochastic Control: A Bayesian Nonparametric Approach
• A Multilevel Approach for Stochastic Nonlinear Optimal Control
• Abstraction in decision-makers with limited information processing capabilities
• Hierarchical Linearly-Solvable Markov Decision Problems
• Experimentally achieving minimal dissipation via thermodynamically optimal transport
• Fast rates for online learning in Linearly Solvable Markov Decision Processes
• Globally Optimal Hierarchical Reinforcement Learning for Linearly-Solvable Markov Decision Processes
• A fictitious-play finite-difference method for linearly solvable mean field games
• A Bayesian perspective on classical control
• Near Time-Optimal Hybrid Motion Planning for Timber Cranes
• Universal Convexification via Risk-Aversion
• Goal Kernel Planning: Linearly-Solvable Non-Markovian Policies for Logical Tasks with Goal-Conditioned Options
• Suboptimal Stabilizing Controllers for Linearly Solvable System
• Action selection in growing state spaces: Control of Network Structure Growth
• Connection among stochastic Hamilton-Jacobi-Bellman equation, path-integral, and Koopman operator on nonlinear stochastic optimal control
• Nonlinear stochastic receding horizon control: stability, robustness and Monte Carlo methods for control approximation
• Noise equals control
• Adaptive Smoothing Path Integral Control
• Path integral formulation of stochastic optimal control with generalized costs
• Iterative control strategies for non-linear systems
• Deterministic particle flows for constraining SDEs
• Semidefinite Relaxations for Stochastic Optimal Control Policies
• Information Theoretic Model Predictive Control on Jump Diffusion Processes
• Risk Sensitive Path Integral Control for Infinite Horizon Problem Formulations
• Implicit sampling for path integral control, Monte Carlo localization, and SLAM
• Canonical Cortical Circuits and the Duality of Bayesian Inference and Optimal Control
• Stochastic quantization approach to modeling of constrained Ito processes in discrete time
• Nonparametric Infinite Horizon Kullback-Leibler Stochastic Control
• Deterministic particle flows for constraining stochastic nonlinear systems
• Hierarchy through Composition with Linearly Solvable Markov Decision Processes
• Calculating principal eigen-functions of non-negative integral kernels: particle approximations and applications
• Information, Utility & Bounded Rationality